To take a more nuanced view, I think there is an important distinction, frequently lost, between "can't code" -- which is all too common in practice -- and "can't easily code a stream-of-consciousness solution to a synthetic problem unrelated to anything I've ever built". Or its close cousin "can't easily code a toy solution to your toy problem since I've only worked on massively scalable versions of the same problem". Something I keep in mind when interviewing candidates is that there is that good understanding of the general design of algorithms does not imply that they should be able to throw together an implementation of an arbitrary algorithm with minimal effort unless they've needed to do it in the recent past. Expecting people to spending many days practicing the implementation of arbitrary and often irrelevant algorithms is unreasonably in my opinion. I am more interested in figuring out if they could given adequate time.
A valuable practice, which surprisingly few tech companies do, is to ask candidates about diverse and orthogonal problem domains, or alternatively, allow the candidate to pick from a diverse set of problem domains. In the former case, you often find that candidates have difficulty writing even simple solutions for some problems but on others they instantly and fluidly can code up a good solution. Because they've had to do it in recent memory and still have the "muscle memory" from doing it previously. You often see bipolar results across the problem set this way.
For senior engineers with deep domain expertise in an area, there is an additional trap in that they use more sophisticated and often very different algorithms in their day to day lives than are applicable to the toy problem domains. Graph algorithms are a good example of this, and they are popular in interviews. The representations most engineers know (e.g. adjacency list or matrix) don't scale but the extremely high scale algorithms operate on a different set of principles that aren't trivial to code and aren't relevant to non-parallel cases; coding up an adjacency list graph traversal algorithm is going to be very unnatural to a software engineer used to doing the same on trillions of edges in real-time. It would be crazy not to hire an engineer on this basis but I've seen it happen, ironically because their expertise caused them to show poorly on the coding exercise.
Interviewing for technical skills is intrinsically difficult but I think that as an industry we are much worse at it than we could be. For all the claims of companies that they only hire the "top 10%" or whatever of engineers, the interview process is often optimized to the benefit of the median engineer.
Most engineers would not hire themselves. That has been apparent to me for awhile now. I’m not sure why they expect people to be to be better than they were when they were hired. I don’t expect engineers to be better than me. I have but one qualification. Can they do the job? Are they strong enough that I can guide them into the position I need them at if it is required.
So much focus has been put on 10x this and high performing that. Companies waste lots of engineering talent and company money looking for 10x when 10x is mostly situational in nature. Set the scene. In one scene the engineers are not producing. Change the scene and now they are 10x.
Instead of looking for 10x, that time would be better spent making incoming and current engineers better at their jobs and creating 10x situations.
> Companies waste lots of engineering talent and company money looking for 10x when 10x is mostly situational in nature. ... In one scene the engineers are not producing. Change the scene and now they are 10x
I can relate to this. I have pretty good resume - good schools, advanced degree, impressive sounding projects, a long list of publications. My track-record suggests I am at least a 3x engineer.
So when I get into a coding interview, I'm expected to perform as such... but I usually come off as a 0.3x engineer. Like really bad. Forget syntax bad. Forget algorithms bad. I've bombed so bad in a technical interview, the interviewer questioned if I even knew basic trigonometry. They had spent 10 hours interviewing me prior and felt good, but got me in person and suddenly thought I was an idiot.
In each of my jobs, I've had a considerable ramp-up period. I'm talking 6 months or more of feeling completely lost and unproductive. I get in my head and it makes me perform worse.
If I'm not fired in that first period, I'm lucky. Only after about a year on the job I can perform at the level my resume would suggest.
I love to flip that around in an interview before it even comes up, one of the 3 things I start out any interview with is:
"I'm your Google, anything you'd need to look up just ask me and I'll give you the answer".
Great way to take the pressure off plus it gives you a good idea on how well they ask questions/are comfortable leaning on others. It's a huge plus if I can get a reasonable candidate to admit that there might be parts of the problem at hand they don't understand.
This is fantastic. Often I am wondering should we start test how good engineers are in asking questions for search engines. I am not going to write the perfect compilable code for them, I have a tools to tell me if I forgot the semicolon in line 31. I don't need to remember standard library 100% and recall all functions' signatures - I have a docs for that. But I should be able to ask meaningful questions that get me closer to the answer and I should do it efficiently. We have tools now, we should start using it instead of memorizing things.
"I'm not a reference manual." I'm going try to remember that to use in an interview some time when I don't know something off the top of my head, and they get in my face for it.
You probably wouldn’t want to work anywhere where people get in your face for not knowing something off the top of your head. The three phrases I wish that I heard more frequently at work are “I don’t know”, “I’m not sure” and “Do you have any ideas?”
I've written code tests where I'm expected to read the code, find the issues and fix them on paper. But when they take it too far beyond the spirit of what I believe to be a reasonable test of understanding and fixing code, then I write that I'd just for example compile the program and look at the output.
The companies that are most into questions like these seem to be the classic (failing agile) body shops with ridiculous turnover of their talent, which causes them to have to interview even more, which causes in turn for their tests to become even more abstract and worthless.
I have a long text file that I copy and paste out of that has pretty much everything I ever do from read/write from databases to displaying data 90% of everything I do is a variation of the same thing.
Forgetting syntax shouldn't be a problem. That's what manuals are for. Of course, if you are applying for a Java programmer job and you don't know the first thing about how Java program looks like, it's a no-no. But if you forgot some detail about how to write some obscure construct - that's what manuals and SO are for.
6 months to get into things is kinda long though. Unless it's super complex things, it should be less. Are you sure you are not exaggerating?
There's a difference between "feeling unproductive" and actually being unproductive. The person you're responding to said the former, not the latter. And in my experience it's nearly universal that when you ask excellent and very senior engineers "how long did it take you to feel ramped up?" they answer "I still don't!" despite having been there for months/years.
There’s a reason people refer to it as “ramping up”, as you become familiar with the environment and code base you become more and more productive until more or less plateauing a while later. It’s not really useful to categorise it as binary productive or not productive.
I respect your opinion but I find this strategy very strange. (Or maybe I don't exactly understand what you're saying.)
In my view, it would be a dream scenario if the next 10 programmers I hire were all superior to me. If I was the worst programmer on the team, that would be an ideal outcome. Yes, I've been programming for 30+ years but I'm also self-aware of my limitations. This gives me the ability to recognize better programmers and learn from them. If I'm not hiring interns, I do expect (or at least hope) that they are better than me.
Am I competent enough to write a "do/while" loop and knock out FizzBuzz in 5 minutes?!? Sure, but that doesn't mean there aren't better programmers than me. I would consider it a great business skill if I'm able to consistently attract superior programmers -- either to work with me or for me.
Larry Page attracted programmers better than him (e.g. Jeff Dean). Yes, Larry earned a masters degree in computer science but his java programming (the rudimentary 1996 crawler) wasn't considered very good. His employees rewrote the whole thing in C++ and made it scale for billions of pages. Bill Gates hired superior programmers. Mark Zuckerberg hired superior programmers. Etc. etc.
You actually agree with each other. You're saying "it would be a dream scenario if the next 10 programmers I hire were all superior to me"; it's an ideal outcome, even if unlikely. The parent's "I don’t expect engineers to be better than me" merely points out the unlikeliness, you point out the desirability.
When I was at one company (later acquired by Google), one of the developers created a model to show just how unlikely it was that every new hire would continue to raise the average level of competence for the team as a whole (as our hiring traditions espoused). After years of hiring very good programmers, it just wasn't going to be possible to hire many more who could pass that test.
It doesn't seem like it. The next 3 sentences from the thegayngler:
> I have but one qualification. Can they do the job? Are they strong enough that I can guide them into the position I need them at if it is required.
In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them. They can come up with programming solutions I could never think of. It's very realistic that "the qualification to do the job" essentially means they are a better programmer than me.
> In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them.
Presumably, in your mind, "programming" skill is along one spectrum and one focus, as well? I think that's part of what's being talked about here. There are many, many aspects to software engineering/programming (depending on whether you want to classify them differently), and many roads that could have been taken to get to the point someone is at. I see no reason to think that someone that is better or worse than me in one aspect will also be so in every one of the other aspects of the job.
>There are many, many aspects to software engineering/programming [...] I see no reason to think that someone that is better or worse than me in one aspect will also be so in every one of the other aspects of the job.
Let me try to restate that in more general terms and you can tell me if I interpreted it correctly: Since programming skill is multi-dimensional / multi-faceted, and it is unlikely for a programmer to be superior in _all_ dimensions compared to another, it is therefore not possible to conclude if one programmer is superior to another.
Is that a fair rewording?
My response is... sure, programming skill can be looked at as multi-dimensional. But so can other real-world comparisons. We prioritize the dimensions that are more important and collapse all aspects into a composite score.
Multidimensions such as choosing a new job: Company X is a 5-minute commute with 4 weeks vacation but Company Y has the newest technology and has lucrative stock options. Yes, it already goes without saying that there's more than 1 aspect of "desirable working conditions" at each company and no single company can claim a superior score on _all_ aspects. We can still weigh all aspects and eventually pick one (and only one) which is superior.
If I hired a "Jeff Dean" type of programmer, it would be an upgrade to my team and my company. Could I search for an "aspect" that I'm better at than him? Well, looking over his expertise[1], it looks like he doesn't have much frontend GUI programming experience. So yes, I guess I could claim superior skill on writing event handlers in C# language for buttons. That's not important though for my assessment. My point is I don't need for programmers to be superior to me in every aspect for me to determine that they are superior overall. (Same as determining which company is "superior" to work for.)
If I was founding a startup and I was the "ceiling" of programming skill instead of the minimum "baseline", my company would fail. I think I'm pretty decent at programming but I have no Dunning-Kruger delusions that I can "train" anybody that can pass FizzBuzz into a "Jeff Dean". I know of no 6-week bootcamp or even a 6-month corporate training program that can claim that feat. What I can do is recognize the programming skills in others that are (overall) superior to mine and hopefully convince the candidates to come work with me.
Well... no. But I think that's my fault, for the most part. I read the part of your original comment that said 'In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them.' as implying all their abilities as being beyond you, but it doesn't specifically say that.
I wasn't trying to imply that it's foolish to try to ascertain if one programmer is generally better than another (as long as some common domains along which to measure are involved), just to note that it's possible to hire someone that's far beyond you in some areas, and behind you in others, with the intention of specifically helping them build in the areas they are lacking (which have application to the job). The result should be a programmer that is, generally, as good or better than you in most areas, making them a more obviously superior programmer, even if that wasn't entirely obvious at the time of hiring.
In short, you can hire for the things that are important that you can't easily improve while allowing deficiencies in the areas that are less important which you feel you can help them improve in.
As a strategy, it has it's pros and cons. As a pro, it's probably easier to find candidates since your criteria is less stringent. As a con, you might find the candidate you hired has problems coming up to speed in those areas they are deficient in (which may be why they were deficient in the first place, instead of lack of experience).
I didn't really express the point very well originally, and just stopped at the opening, probing question.
Of course you want the best you can get. I’m arguing 10x is constantly influx for any number of reasons. So it’s unlikely you’ll get one. If you have a process for making engineers into the best then you don’t need to look for the unlikely 10x because you can make them into the 10x high performers you need.
Ive seen companies literally turn good engineers into underperforming engineers...stifling them with process, tech debt, politics, etc... Then they get angry some people can perform and others can’t under the circumstances and conditions the management created.
I’ve watched perfectly good engineers get fired or forced out for not being miracle workers only for people to later realize there was nothing wrong with the engineer in the first place.
You don’t know who is going to be the best until you work with them. I would advise people to pick people they will like even under the worst of conditions and train them to the way you want them to be.
I have observed exactly the same thing. It's truly mindblowing.
I also find it remarkable how quick people are to blame the engineer, rather than considering the whole picture (process, tech debt, politics, etc). It's a complicated equation with a large number of variables.
I recall one of my friends, when I was at Google, sending me a screenshot of code written by Jeff Dean with an obvious bug. Jeff is, of course, extraordinary, but I wonder how many companies might have passed on him if he submitted that code in an interview setting.
Absolutely. There exists no single human being that does not make any mistakes. The trick is to find the people that make as few as possible, know how to track them down, and that can figure out how to hot-fix them as needed.
Java in 1996 was version 1! Doesn't necessarily say much about Larry'd code-foo :)
Jokes aside; I agree with the advice of hiring people smarter than you in the domains where you won't be paying attention should the company actually take off.
If you're looking to be a founder/CEO level type, doesn't seem to matter what your skill in Java and OOP are, yes? No need to be a water walker.
That's the "genius" of management; they sell an idea and get someone else to do the work.
I noticed that quite a few programmers have the following attitude: If there is something (a programming language, framework, library, paradigm, design pattern, tool...) that they know and use, then they believe that familiarity with it is essential to call yourself "senior". On the other hand, if they do not use it, because they do not know it, they believe that it is useless and anyone studying it is just wasting their time. They may not be aware that this is the algorithm they are using to evaluate others.
Imagine a programming ecosystem with e.g. 5 approximately equally popular frameworks. Suppose that someone with this attitude is familiar with frameworks F1 and F2, but never used F3, F4, F5. Put such person in charge of an interview, where equally skilled candidates (i.e. knowing 2 of the 5 frameworks) apply. They will only classify 10% of them as "senior-level" (those knowing F1+F2), and additional 60% as "somewhere between junior and senior level" (those knowing one of F1,F2; and one of F3,F4,F5). So they would hire literally themselves, but probably not someone on the same level.
IIRC a Google recruiter once sent a set of packets to a hiring committee and all of them got a "no hire". Then the recruiter revealed the packets were lightly-anonymized versions of the actual packets of the hiring committee members, from when they'd all originally interviewed.
Agreed. I wouldn't even mind so much if they picked "better for the job". So often, it seems like the criterion is "better at doing things that never actually come up in the job".
Productivity follows when people are working for a worthy cause. In fact the core of productivity isn't lists, management techniques or whatever, it is a thing called 'constancy of purpose'
There exist no such a thing called 10x burger flipper.
"A players hire A players and B players hire C players"
- Steve Jobs (Maybe?)
If you don't encourage a culture where everyone is asked to hire people better than themselves, it can easily degrade where each progressive hire is slightly worse.
It also entirely depends on your career path. Mine didn't involve programming outside of shell for most of it, and involved tons of deep expertise on a variety of topics.
I was shocked doing an interview in my last round and got so many test questions. I bombed so hard because it wasn't what I was doing.
The real key to me is, we treat all of these jobs as more or less the same, when in reality they are all vastly different. There is no singular senior engineer skill set, but a variety.
Also, expectations of coding tests should be upfront. E.g. we're looking for the most optimal solution, not the one that is most convient.
I was declined for a position at a trading shop for a C# backend role because I used linq to do something. I didn't have unlimited time for the tests, so I went with expedient over performant, for which they did not care. Fuck that, set clear expectations for interview "tests".
If you tell me "here, do this, you have 2 hours" I won't even think about the possibility that you want anything that isn't quick to write.
If your priorities aren't aligned with the environment you set up, you are the one that must say so. On a real world project that means once in a while I'll take a project back for small fixes, instead of wasting unending hours thinking about every possible problem up-front... if you expect something different, well, that's not optimal.
You also need to ask about resource time and money budgets here some little start up is not going to have the resources a tier one TLA will have in its black data centre.
> For all the claims of companies that they only hire the "top 10%" or whatever of engineers, the interview process is often optimized to the benefit of the median engineer.
This really seems out of necessity. There are going to be a lot more qualified median engineers rather than superstar coders, so it makes sense to optimize your hiring pipeline to assess these folks.
In my experience, most of the "top 10%" hires are either through interpersonal contacts or a fast-tracked interview process for a specific candidate.
I can just imagine a room full of top 10% engineers/architects...No we need to do it like this...Well, no did you think of this obscure use case... what about performance? I think...should we...fast forward a year later and the company went belly up because everyone was trying to be best and piss on each other to prove who was top...top.
At least when middle managers do it they're only interested in who gets credit for the work so they leave the average tech guys alone long enough to get the work done.
OTOH, I've had the good fortune of working in a team of top 10% engineers/architects (I'm not top 10%, btw) who worked harmoniously, and it was one of the best times I ever had in my career. Learned a lot, upped my level trying to keep up, helped create a good product, etc.
In that case, the key was that clumps of them had already worked together in earlier jobs, so the incompatibilities had been pruned.
I've worked at a company that generally hired extremely good engineers - much better than average and they were very successful because (1) doing that was actually important to the very hard problems they were solving, and (2) every other company in the world wasn't trying to take the same approach to hiring. Today, everyone thinks they should hire like google, even though they're building a mid-sized web or phone app.
Really hard puzzles that you had to submit a solution to if you wanted an interview. This was very rare in those days. The problems were original and clever, not pulled from a book or website. Also, the company was in Cambridge, MA and had very close ties to MIT so they could find people like that.
If you got an interview, and did well you got another challenging programming problem to solve on-site. You were given a computer with the programming environment of your choice and as much time as you wanted, as well as someone to ask questions from.
I think it was also important that the company solved hard graph problems in Lisp, with a lot of macro-driven code. If you were a Lisp programmer, you generally wanted to work there.
Someone being smart does not mean they are going to be an architecture astronaut or that they will be unable to cooperate with other people (especially with people who are roughly as smart).
Also depends on what they're counting in their percentiles.
You can pretty easily hire the top 1% of applicants, and still get only mediocre results. E.g. at one job, when looking for a senior-level eng position, clearly stated in the description, we easily weeded out a couple hundred applicants who had effectively zero experience whatsoever. No prior programming-related jobs, internships, side projects, open source, nothing.
Which is a waste of time for everyone involved. Hence the heavy reliance on connections :(
Hmm, maybe we should 'equalize' the playing field then: Give everyone something from left field that pretty much all people bomb, that'll see how quick they learn.
Exp: You must code in an unfamiliar language (Fortran 1988, for example) and do something simple in it. At least then you'll know that people are all starting from scratch. Time it, make sure it doesn't last more than 2 hours, see how laughably slow we all are, pick those that get the furthest.
I've worked somewhere where we used this approach for one of the 5 interview sections.
Even being upfront about our low expectations and providing very friendly and helpful people to pair and guide, it lead to record numbers flipping the table and stomping off premises before the completion of the interview.
I don't use this approach now, as I don't think its a good use of time (and I think an interview should be a positive experience even for those that fail), but it did provide, I believe, a fairly strong signal of the attitude with which a candidate approached learning new things (but not really anything else).
Wow, alright, this is exactly what I was thinking. The 'novel coding' test was just one test, that it was forewarned to be laughably hard, and that you all were friendly about it. I really do want to know more about how this all went as I thought this would be a really good way to interview coders, but unfortunately I am totally wrong. I want to know why my thoughts are totally off-base.
> I thought this would be a really good way to interview coders, but unfortunately I am totally wrong.
It's not that you're wrong, it absolutely is a really good way of finding out how someone reacts to being asked to learn something new in a stressful situation. And that is a valuable thing to know - I love working with people who get excited about an opportunity to learn something.
It's just that you have to be prepared to give so much help and the question has to be so easy because it's all completely new that you can't test competence in any meaningful way. Which means that at the end of the 90 minutes or whatever, you know exactly one thing more - their attitude to learning under pressure, but you have relatively little time with them, and you probably need to learn multiple things per session.
Maybe this cockamamie way of doing interviews is the correct one then? Each company tries their own thing, and if you get flustered and bomb it, we'll both parties know it not a good fit. Some companies value the ability to learn and be cool under fire, some like GPAs, some like HW tests. The sorting occurs and is not 'fun' but works in it's own way.
I like to pose technical problems that can scale to the candidate's level of experience. If I sense I'm hitting the candidate's comfort level, I can back off. Otherwise, I can take the problem even deeper with more technical questions. Either way the candidate still has a positive experience.
This is my approach as well. Ask really easy questions and get more complicated as they do well. I also try to increase difficulty in line with the strengths espoused on their resumes. If they are fantastic with databases, I might make sure the problem can be solved well with a database oriented solution.
> Even being upfront about our low expectations and providing very friendly and helpful people to pair and guide, it lead to record numbers flipping the table and stomping off premises before the completion of the interview.
You also have to be careful when in the interview you throw this. A bad hit like this can throw a candidate off for the rest of the interview even if they are quite acceptable.
That would still not level the playing field for several reasons:
1) Some people may find Fortran easier to understand than others even if they haven't touched it before. This was the case when my University decided to use Haskell* as the first language to teach students in order to level the playing field. It did help somewhat, it leveled the field for those with little and those without any prior programming knowledge. But to those who had already come to terms with recursion etc it was a very tiny stumbling block.
2) Some people need to dive deeper and gain a greater understanding of things before they feel at ease using them, but once they have reached that level, they will often outperform most other people.
* This was 20+ yrs ago and Haskell was not so well known back then.
I’d probably pick the ones who interrupt the coding exercise, tell me how ridiculous it is, and tell me about the ways they’d solve it in plain English / pseudo code.
I love this idea. Testing how well someone can learn new skills can be just as important as the skills they've already learned. However, the problem is that as soon as it becomes known that Google is giving tests in Fortran 1988, applicants will study Fotran 1988 and thereby reduce the efficacy of the test. It will also have the sad side effect of incentivizing people to waste time studying Fortran 1988 rather than building skills useful to their job. It seems to be a good interview needs to satisfy the following properties:
* Predictive of job performance
* Either (1) difficult to optimize against or (2) easy to optimize against but in a way that doesn't reduce predictive value
> However, the problem is that as soon as it becomes known that Google is giving tests in Fortran 1988, applicants will study Fotran 1988 and thereby reduce the efficacy of the test.
How about "Google is giving tests in these 25 languages"? They could even choose languages that are purposefully dissimilar from any language that you've claimed familiarity with.
What are the graph representations that are used at larger scales? They're not adjacency lists at bottom? Can you point toward some resources to read up on?
Would it be better to have the interviewer provide take home context a few days before the interview that will be related to the coding problems during the interview? Somewhere between the take home coding problem and the whimsical toy problem solving by during the interview?
I really don't like take-home assignments. It's a one-sided time investment. If I'm asked to do one I say that I'm only prepared to put in 45 minutes on it. Now that I've done this a few times and subsequently been rejected after submitting, I'm leaning towards rejecting assignments altogether. It doesn't really serve me well in job searching but I feel strongly about it. It's a bad investment on my time and the assignments never asses things I'm actually good at.
Same. Now that I am on the other side I send out a short piece of code and ask the person to point out all the errors in it. There are about 20. It takes about 5-10 minutes of the candidates time and I have had a lot of success with it.
Really wish more interviews were like this. My favorite onsite interview was when the interviewer printed out a class from the actual system I'd be working on (as I verified for myself later), asked me to figure out what it did, any errors I found, and a few ideas to improve or refactor it. Then he took my code sample and asked me to describe what it did as well.
After that he said "Okay, I know you know enough to handle the job, now I'm curious how much you really know. " (This is key, I feel. He specified I wasn't going to get dinged for not knowing the answers, so I didn't feel the pressure in the questions he asked afterwards, when I usually feel intense pressure, as I know I've been passed over for getting a single question wrong in multiple interviews in the past.) He then asked me some pretty deep questions about low level memory and other things, and if I said I didn't know he turned into teacher mode and taught me the concept.
I actually walked out of that interview having learned something new and useful. I then got to chat with the President very casually, and I received a job offer a few days later.
The interviewer new his stuff, too. He'd been programming arcade games in assembly for decades. His games have sold millions of dollars, and two of the series still get made today.
Ever since, I figured if a freaking legend could be satisfied after reviewing some code and a code sample, the crap I've had to endure everywhere else is completely and totally unnecessary, and it's just frustrated me to no end.
I had a very similar experience many years ago. It was an start-up. The interviewer was the ex-CTO of a widely used open source project/product, and led the engineering team in the start-up. Although I regarded myself a good engineer, I was pretty nervous.
After several questions that he was sure I knew the regular stuffs and was qualified for the job, he said he was going to ask some really hard questions to see how much I really knew and what was my thought process. The questions had no simple answers. Each of them required knowledge and experiences from a few technical fields. I didn't know much then. He guided me how to solve the problems, discussed the pros and cons of each approach. Then he told me how they did it, how other products in the same category did it, etc.
The half-hour interview was prolonged to one and a half. At the end, he said, "You have a very intuition on what are the right directions to go when facing unknown complex issues", and gave me an offer. It is my best interview experience. (I didn't join the start-up due to other reasons.)
The technologies have been advanced in a blazing fast speed i the last 10 plus years. Technical interviews, on the contrary, regress in a similar speed.
Not a take home assignment, just prep material. Like saying, we will ask you about X, Y, and Z, here are some pointers. I would just worry the night before anyways, might as well put my anxiety at ease by prepping and building confidence.
exactly. If the job and the pay were exceptional, sure. But today it's standard practice for average companies.
One company I interviewed with gave me a program that had to be written in Scala, with the full knowledge that I would have to learn Scala to do it. Please.
> ...the assignments never asses things I'm actually good at.
I actually really like homework assignment approach when done correctly, both as an interviewer and as an interviewee (though seanmcdirmid's suggestion of prep material, rather than a homework assignment, is quite sound), but this is a separate issue, independent of how skills are assessed.
It's really about the company/team having a preconceived notion of what skills they are looking for vs. looking to see if a candidate has a distinctive skillset that could help round-out the team. How do you, as the candidate, market yourself and the things you are good at?
Obviously, the resume is a good place to do such marketing (and I'm sure you already are) so if teams are turning down applications based on the homework without looking at the other skills the candidate has to offer, then that might be an issue.
Take-home assignments are a great way to filter out women, the sick, and the poor, but make it look totally above board.
"The IT industry is committed to increasing the number of women engineers. But if they're too busy caring for their children to spend six hours doing our silly test, that's their fault for being women. We are forced to hire 20-something white males who have nothing better to do with their time."
There are us guys who are involved in toddler care also (like me, since my wife is working instead). Anecdotally, my wife failed in all her in person interviews until she got one with a substantial take home assignment. She got an offer for that job easily, above what she was looking for.
For her at least, this worked out better for her. She does design, which is even more difficult to evaluate in 30 minute time slots.
Not that I’m arguing this is the way to go, I was just wondering if interviewers could give some hints on what they would cover I the interview, so we could prep like studying for a test.
> Take-home assignments are a great way to filter out women, the sick, and the poor, but make it look totally above board.
I hope your not impugning the motives of those who issue take-home assignments. Existing practices for hiring are objectively bad at selecting qualified candidates, and most people are just trying to do better. There is some evidence that take-home assignments are an improvement above the norm.
Also, consider that such assignments can serve as a "blind-audition" and can help eliminate bias from the interviewing process. And there are lots of ways in which loading everything into the interview unfairly penalizes certain types of otherwise qualified candidates. What's left, referrals? That's definitely biased, even if it also tends to have a lower false positive rate.
Motives are mostly irrelevant, if the effect is the same.
An easy switch would be to convert take homes to onsite work. One of the more recent interviews I had consisted of putting me in a room with some data and a stub of code and asked me to fill in the stub over a few hours. Every so often, they’d come by and ask if there were any problems. (More frequently at first, and then later less frequently.)
It’s basically the same, but it’s timeboxed and well defined with enough back and forth, to not feel like they’re just throwing a problem over the wall to you.
They are relevant enough that the prior poster felt the need to smear them.
> if the effect is the same
The claimed effect is, at this point, purely hypothetical and indeed runs contrary to my experience.
> An easy switch would be to convert take homes to onsite work.
Generally speaking, an onsite task like you describe seems like a reasonable choice for a company to make. However, this strikes me as worse for the affected groups that reaperducer was concerned about:
- It requires scheduling an even larger fixed block of time outside one's normal job/life responsibilities. A take-home assignment at least allows for flexibility for when you work on it. It strikes me that time-poor candidates would appreciate the flexibility.
- It would no longer be a "blind audition". When I've reviewed take-home assignments, I don't know anything about the candidate except that the code. It strikes me as a good way to resist unconscious bias from seeping in.
I think you’re reading malevolence where none is intended. Simply pointing unintended consequences is not smearing. While it is true that the effect is merely claimed, as I know of no relevant study, your anecdote equally meaningless.
The the on-site solution remains a blind audition if the people doing the evaluation are not the people administering the test. This is exactly the case for a take home assignment, only now it is timeboxed and with access relatively timely help if needed.
The purpose of the blind audition is not to hide the identity of person being interviewed, as much as it is to enforce an objective standard and remove bias. If the rubric is, “Accept all that got automatic evaluation score of at least X”, then that achieves a similar goal.
As far as time-poor people, never underestimate the desire of having a clearly defined block of time. Of course, onsite or not can always be optional as well.
> I think you’re reading malevolence where none is intended. Simply pointing unintended consequences is not smearing.
Pointing out (potential) unintended consequences is fine, even welcome.
Putting sexist remarks in the mouths of others is a smear, even if it is done merely flippantly.
> While it is true that the effect is merely claimed, as I know of no relevant study, your anecdote equally meaningless.
Anecdote is more meaningful than mere speculation. Even without statistically valid data, examples that are contrary to our expectations can bring to light factors that we had not previously considered.
> The the on-site solution remains a blind audition if the people doing the evaluation are not the people administering the test...The purpose of the blind audition is not to hide the identity of person being interviewed, as much as it is to enforce an objective standard and remove bias. If the rubric is, “Accept all that got automatic evaluation score of at least X”, then that achieves a similar goal.
A fair point.
> This is exactly the case for a take home assignment, only now it is timeboxed and with access relatively timely help if needed.
The point about timely help is definitely a point in favor of on-site work tests. But working in a timebox can also be stressful. A take-home assignment without artificial deadlines (i.e. don't schedule any interviews until after the assignment was received and approved) can eliminate that stress.
> As far as time-poor people, never underestimate the desire of having a clearly defined block of time. Of course, onsite or not can always be optional as well.
I think this hits on an important point, and one that I think would benefit from research and expertise I don't have. I'd hazard a guess that it depends upon why one is time poor.
If the candidate is intelligent and capable, but somewhat disorderly, then they would probably benefit from a structured, on-site task.
If the candidate is more organized but simply has a difficult and rigid schedule (which is actually something I would expect true, for example, of a working mother) then a take-home assignment might be better.
I don't know that either one is superior in all cases. Honestly, I think the best option would be to give the candidate a choice.
Blind audition is BS, because you're never ever comparing apples to apples, and frequently because the problem set is little like how you actually work. If the end goal is to find someone who can do the work well, be a pleasure to work with, and contribute to your organization or company in a meaningful way, we have not optimized for that. We've optimized to hire kids out of college who are experienced taking tests, or for people who have worked on the exact same stack you've worked on.
You will never be unbiased in a hiring process, because the bar you set is not universal, and always be biased in some way. Being "as blind as possible" isn't the right metric, and we shouldn't hire based on it.
Geez I could not agree more. Not only has it, overall, been a bad investment of time (for myself and others I know around me in the field) but so many companies are unclear in what they're looking for in that test to begin with.
I wrote a thing about it and I got someone who responded to me with a rather scathing rebuttal. I don't know, it's very conflicting.
It's polarizing. Everyone has their own viewpoint and it feels like that's the viewpoint they cling to and so it becomes personal when you go against it. Something like that, anyways.
I feel like take home assignments waste less of my time than an interview that doesn't properly assess my abilities... They're also fantastic for junior devs who can prove themselves without extensive working history.
I'd expect that interviewee to request reschedule then. Isn't that the job of the recruiter to manage the interview process and ensure that requirements like this are being met?
I usually like sending take home assignment for graduates.And having a discussion about the approach taken, and do a peer programming session to add features.
However, don't see how well take home can work for senior engineers. In fact, its unfair and delays interview process to expect take home assignments from working folks.
A valuable practice, which surprisingly few tech companies do, is to ask candidates about diverse and orthogonal problem domains, or alternatively, allow the candidate to pick from a diverse set of problem domains. In the former case, you often find that candidates have difficulty writing even simple solutions for some problems but on others they instantly and fluidly can code up a good solution. Because they've had to do it in recent memory and still have the "muscle memory" from doing it previously. You often see bipolar results across the problem set this way.
For senior engineers with deep domain expertise in an area, there is an additional trap in that they use more sophisticated and often very different algorithms in their day to day lives than are applicable to the toy problem domains. Graph algorithms are a good example of this, and they are popular in interviews. The representations most engineers know (e.g. adjacency list or matrix) don't scale but the extremely high scale algorithms operate on a different set of principles that aren't trivial to code and aren't relevant to non-parallel cases; coding up an adjacency list graph traversal algorithm is going to be very unnatural to a software engineer used to doing the same on trillions of edges in real-time. It would be crazy not to hire an engineer on this basis but I've seen it happen, ironically because their expertise caused them to show poorly on the coding exercise.
Interviewing for technical skills is intrinsically difficult but I think that as an industry we are much worse at it than we could be. For all the claims of companies that they only hire the "top 10%" or whatever of engineers, the interview process is often optimized to the benefit of the median engineer.