My primary criteria when interviewing junior candidates are:
1) Do you have basic problem solving skills?
2) Can you communicate clearly?
3) Do I want to sit next to you for the next 6 months or longer?
If you don't know Ruby, I can teach you. If you don't know Elasticsearch, I can teach you. What I can't and don't have time to teach you is how to solve a problem on your own without me holding your hand, and I especially don't have time to waste trying to communicate poorly with you.
And obviously, I want to work with someone who is pleasant and interesting. I don't wanna sit next to someone for 40 hours a week who stinks or is rude or can't carry a conversation.
If you meet those three criteria, you've beaten 90% of candidates that walk through the door. The last 10% is what gets you the job. (Obviously, if you DO know Ruby or Elasticsearch, that's a huge plus... but it's not one of the bare minimum requirements.)
As a junior developer, I find myself confused by your comment. It seems as if a large proportion of the posts on HN claim that, when looking to hire someone, they focus on problem solving abilities and and culture fit.
However, in May when I was interviewing with companies (including YC backed startups), almost every company focused on quizzing me about trivia [1]. I was actually given a paper quiz by one of the companies on equality comparisons in Javascript (e.g. "Does 4 == '4'?"), which I found pointless, as I can test that within seconds at a computer.
[1] The major exception was 42Debut. Even though they didn't give me an offer (just stopped replying to my emails, but c'est la vie), I was treated with respect throughout the whole process, and the interview was very well done, with challenging questions that forced me to think, and didn't rely on language trivia. I would recommend the company to any of my friends, whereas I can't say that about almost any other company I interviewed with.
That kind of question can actually be useful as a proxy for experience with the language, and whether you're the type of person who digs into understanding your tools. If you say "I'm a rockstar Javascript dev", it's very hard to prove it; but if you've never been bitten by some of Javascript's type conversions, and/or have never really looked into some of its intricacies on your own, you probably aren't. It's just one data point, it won't be enough on its own to sway the interview either way, but it's interesting.
To give you an idea, I once interviewed a Javascript dev who said you should null out your variables with 'undefined' when you were finished with them. Oh, why is that? 'Because sometimes they leak out into the global scope and that can cause problems elsewhere in the code.' This person had noted that you could declare a global anywhere in Javascript, but had not noticed, nor asked anyone, nor looked up, nor anything, to find out that there's a difference between "x = 5" and "var x = 5". We didn't hire him.
Had he simply been unaware you -could- declare globals (and had instead said that "x = 5" won't work, you have to use the 'var' key word, or something), we wouldn't have counted it against him; he's following a best practice and never encountered it, fair enough. If he had never been bitten by the scoping rules, fair enough, a minor ding but we'd continue the interview. But the fact he had seen it, possibly been bitten by it, and just handwaved it away as non-deterministic weirdness that required a mystical incantation to avoid? No.
I find it hard to believe anyone has spent a bunch of time writing js and never been bitten by a type conversion. Am I wildly off base?
I do data science, but everyone who writes R has been bitten by it's scoping rules, which are definitely not mainstream. Everyone I know who writes java or python has been bitten at least once by the pass object references by value (is there a better name for this?) function argument semantics. Everyone who writes java has accidentally written == when they meant .equals and stared at the code for far too long. etc etc etc You just can't claim you can competently use a language without understanding some basics, like how functions work, how arguments are passed, and how equality is decided.
Which is the reason I would be deeply annoyed by the 4 == '4' question. The technical answer is it evaluates to true, but the real answer is "any code that requires all readers to have intimate knowledge of Javascript type conversion rules to read and understand is horrific code."
At the same time though, as someone looking for their first dev job, I'm not going to tell the interviewer "don't do that." Given the people I've met in industry, I can't see that ending well- I would expect some variation of "who does this kid think he is!?"
I was treated with respect throughout the whole process, and the interview was very well done, with challenging questions that forced me to think, and didn't rely on language trivia. I would recommend the company to any of my friends, whereas I can't say that about almost any other company I interviewed with.
This is a really important point to remember for interviewers. The interview process isn't just to help you find good candidates, it can be a great branding/marketing vehicle as well. I've gotten many good people from former candidates who enjoyed our process and spoke highly of our team.
I wouldn't have just stopped replying to emails though. An interviewer should always reply, even to politely decline. And it can help to add a reason why you are declining a candidate too, since it can help them improve and perhaps return for another interview in the future (I've hired a few people that I originally passed on too).
Of the many companies that have rejected me, Suitable Technologies (makers of Beam robot doubles) were the best. The biggest thing was that they rejected me as soon as they knew they had rejected me. Within a few hours.
Which hurt, in the short term, because I am a human being. But it was better than many other companies that would say "we'll tell you later today" and then wait a week, or wait for two weeks since my last interview and then reject me in the middle of the CTF they are running and I'm participating in (what, did my answers in the CTF suck?) and then invite me to apply for a job at the end of the CTF after rejecting me. That's really yanking my chain.
"And it can help to add a reason why you are declining a candidate too, since it can help them improve and perhaps return for another interview in the future (I've hired a few people that I originally passed on too)."
I've always tremendously valued the rare feedback I've gotten, but my understanding is this is not the same advice a lawyer would give (and that makes me a little sad).
I don't generally ask those kinds of questions. Mine are more of the algorithmic variety (i.e., given this problem, how do you solve it?). Having said that, the last 10% of getting the job is demonstrating that you know SOME programming. I may ask you some programming questions like the one you presented, just to see whether you know programming at all. If you got this right, you're 95% of the way in the door, as opposed to just 90% of the way in the door. It's not a deal breaker -- I can teach you the difference between 4 and '4' -- but it's a huge plus if you get it right.
Plus, like the article says, most people suck at tech screens, so that interviewer may have just been asking random questions with no clear idea of what they're looking for.
Testing 4=='4' is far from pointless. For better or worse, programming as the majority of us practice it is no longer about flipping bits or moving bytes or red-black trees or building common collections data structures. It's now about a language plus an api or apis, and the language of front end web development is javascript. So checking if front end or fullstack dev candidates have a basic understanding of types in javascript is perfectly reasonable.
One can learn facts. In fact, it is incredibly trivial to do so if you are smart. I need to do it dozens of times a day to do my job, as does any programmer.
I don't program javascript at all, but am vaguely aware of the terrible implicit conversions it does. Isn't that the important part? I do not have memorized that
Why would I (I had to google that, took 5 seconds, btw)? It is ugly, terrible code. Sure, if I spend my days in a terrible code base I might be forced to learn those conversion rules, but that isn't exactly the mark of a hirable person - it is just the employment history of that person.
Now, if somebody who claimed they knew javascript was writing 'j := 0' for assignment I'd be concerned - but probably they've just been writing a lot of Go lately.
I can see a question like this being asked as part of a lead in to a more general discussion about types, type conversion, how different languages manage it and the trade offs thereof. But, most of the nit-picky questions I have ever been asked in an interview were just nit-picky. What is the output of the obfuscated C? What happens if you abuse the syntax like so? Just pointless questions compared to the real thing you should be measuring - can I produce high quality code on schedule and budget that adds value to the organization without making everyone around me wish they were dead rather than talk to me again? Can I recognize inefficieces in the company and improve things? Can I step up and lead a team or a project? If asked to do something outside of my comfort zone, like go on a sales call, can I do it? Will I complain bitterly if asked to fix bugs for awhile? Will I skip off to a new job just as I start becoming productive at your company? Can I mentor junior people? Can I converse with the CEO and tell him difficult facts if necessary? Can others read my code, modify and extend it without undue problems? Can I untangle other people's spaghetti code?
Measure that stuff, not whether I know if 0 == 'false'.
Um, no. If you don't understand types in js, you don't understand js. I'm not hiring a front-end engineer who somehow hasn't learn js unless he or she is a totally raw intern. And a 1st or 2nd year intern, not a 4th. For anyone else, you need to know js, and if you don't understand types, you don't know js. If you need to google your example, you simply can't write js on a daily basis. (The one exception is, as noted below, saying "don't do that, here's why, do this instead"; that's a great answer.)
Types, functions, objects, classes, type conversions, etc, are part of the minimal working set needed to write code in a language.
Well, testing 4 == '4' is pointless in my opinion, because you should never use the '=='-type equality in JavaScript. I never use it, so I don't even know what that evaluates to. You should always use ===, and set up a linter to enforce that.
What I think is valuable to understand is the concept of truthy and falsy, as I've seen even some experienced JavaScript programmers struggle with that. Maybe that could be a decent interview question.
A response of "I'm not sure; I always rely on === and explicit conversions to avoid the ambiguity. My guess would be (doesn't matter) though" is not only answering that question, it's scoring an 11/10 of the possible points (were it pointed).
Criteria #3 is toxic. It's one of those things people adopt with the best intentions, but (a) is often at odds with the best interests of the firm and (b) provides an enormous amount of cover for prejudices and, more perniciously, cognitive biases. When you're interviewing a candidate, assume your brain is trying to trip you up (it is!). Build a hiring process that treats your intuition as an adversary and eliminates cover for biases.
On one hand you have people with immense industry experience (such as yourself) advocating to drop the "do I actually like this person" approach and on the other hand you have people who also have immense industry experience advocating that culture fit should be one of the most important criteria.
I'd be interested in hearing your thoughts on eliminating prejudice (both intentional and unintentional) and cognitive biases whilst at the same time, ascertaining whether or not someone would be comfortable within your companies established culture and also contribute positively to said culture.
To clarify, the difference you aren't seeing between the groups is the incompetent managers define "sit next to" as "sit next to at the nudie bar" or gaming table or harry potter movie premier or frat house or pretty much any place other than work. AKA young inexperienced white frat boys only need apply.
In comparison, competent managers define the same "sit next to" as "sit next to during an emergency bug fix" or while hashing out a new feature idea or optimizing a slow task or automating a process or just gossiping about a new technology. These are the kind of managers who "dare" to hire non-white people or women or older than 25 yrs old etc. Usually their company is much more successful for obvious reasons.
I fail to see how criteria #3 in the parent comment: Do I want to sit next to you for the next 6 months or longer? is assumed to be toxic when it can be as easily assumed the intention is the latter half of your example.
When you hire somebody for your your team, company is not the only one stakeholder. The CEO won't spend 40 hours/week with new hire. Your personal productivity and happiness depends on new hire and if he has (1) bad communication skills (2) unpleasant to be around, it's no hire no matter his technical skills. There are exceptions to this rule of course - like when you won't personally work with the person or he is genius.
I largely agree with B but I'm not so sure about A -- particularly since hiring someone whom you don't want to be around is likely to affect your performance, which is likely also at odds with the best interests of the firm.
For my first job out of uni, I was asked to write pseudocode on a whiteboard to solve a simple scripting problem (call an executable repeatedly, changing the command line args). My background is nuclear engineering, and I was interviewing at a cloud/networking business, so I was already slightly outside my experience comfort zone[0]. I was explaining my thought process while writing on the whiteboard, trying my best to be transparent about how I was thinking, when the interviewer interrupted me.
"What is that?" he asked, pointing at the whiteboard.
"...pseudocode?" I replied, hesitantly, frantically looking for some mistake where he was pointing.
"That's not pseudocode..." he said as he started to berate me for not writing bash.
After that (I did not get an offer), every interview I went to when someone asked me to write pseudocode I'd always clarify "Is there any particular language you want me to use?" because I never want to relive that experience again.
Expecting pseudocode to be in a particular language is objectively wrong. It's annoying that you lost a job opportunity because your interviewer was incompetent, but I would recommend not asking "Is there any particular language you want me to use?" when they ask you to write pseudocode. They might think you don't know what the word means, which isn't that big of a deal, but leaves a bad impression.
I am an instructor at a coding bootcamp and I conduct technical interviews of our applicants pretty regularly. I have recently been very concerned with tackling the idea the author first addresses: not hiring (or in my case admitting) for what people already know. Our program only has 12 weeks with the students and so what I am most concerned with is the applicant's ability to pick up new concepts quickly.
In the past we have tested students on the little bit of Ruby or Javascript they had studied to prepare for the interview. I am of the belief that that method has helped determine who knows a little bit of Ruby but not who can ramp up on complicated topics quickly during the program. My attempts to address this have led me to doing a 15 minute lesson on something totally new to the applicant and then having them answer questions based off of that lesson. So far I've found it to be useful.
Technical interviews are hard. It's easy to suck at them.
Fizzbuzz isn't supposed to be a test of your knowledge of the modulo operator. It's supposed to be a very low bar that can filter out a large percentage of candidates who have no business being there.
If you spend 20 minutes on Fizzbuzz, you're doing it wrong. It should take 10 tops, and be limited by how quickly the candidate can write.
It isn't supposed to be a test of whether you know the modulo operator but in practice it works out that way.
True story; my wife was going over her (correct) answers to a coding test with the interviewer and he asked her "What's this?" in reference to the modulus operator. The interviewer didn't recognize it.
I use the modulus operator almost daily so it seems obvious to me but I realize that everyone has different experiences or things they are ignorant about. IMO, fizzbuzz is a test of whether you know the modulus operator "trick".
> It isn't supposed to be a test of whether you know the modulo operator but in practice it works out that way.
Yep. And at HN, actually knowing this or anything else assures that you will be downvoted with a probability of 1.
> I use the modulus operator almost daily so it seems obvious to me but I realize that everyone has different experiences or things they are ignorant about.
I see the modulo operator applied to a common undergraduate math question -- if a given date lands on a Tuesday, and if you add 500 days, what day of the week does the result land on?
I think FizzBuzz is so popular that if you even needed me to describe the question, I'd be a little confused. I don't think I know any programmers with more than a year or two of experience/schooling that have never seen FizzBuzz before.
Now don't get me wrong, I wouldn't think poorly of someone or of their programming skills because of that. But not knowing about FizzBuzz is like not knowing the name of the current president. Have you been living under a rock for the past 5 years? It seems to be mentioned at least once a week on any sort of newsfeed, forum or blog related to programming.
I'm just about to finish a B.S. in CS, so I've recently been on the other side of the table. I like to think I'm "aware" enough to give good feedback about my experiences.
I've interviewed with two of the large "top" companies. They were two very different experiences.
One decided to have me do multiple interviews, with whiteboard coding. The first interviewer was my favorite, because we got to talk about the design of my internship project, how it could be extended, pitfalls to consider, etc. That actually let me stretch my legs a bit and show that I can make intelligent software decisions. The rest of the interviews were basically worthless; the classic small algorithm problem that I could easily figure out with a bit more time or by working with another more experienced engineer. The code I wrote on the board showed that I could write for loops and use a standard library; it was very difficult to modify or refactor if we saw an issue with what I wrote. Why not at least a laptop, basic text editor, and a projector?
The other company gave me a few hours to write up a solution, in an IDE, to a somewhat beefy problem. We had a couple of discussions about my approach, potential pitfalls, cases that I couldn't handle, optimality, etc. I liked this, since it was much closer to being representative of real software work; collaboration, discussion, and I also got to show how I would actually create a solution.
That's a stark contrast. I'm sure there are flaws in the latter approach, but it is if nothing else a much lesser evil that will likely bring in more valuable engineers.
My favorite interview, ever, was a phone screen. Untimed. They gave me a weekend to solve two not really difficult problems, with well defined interfaces. The catch was to write it in a production ready style. That is, do everything I would do for actual production code. Documentation, test suites, write the code to be as elegant as possible, while failing as cleanly as possible if invariants failed.
I have no idea their results with that, and I ended up taking a job before the in person interview, but I enjoyed the screener a lot.
I'd probably fail that from lack of experience, but it does seem to be a great approach for someone a bit more seasoned. That shows a lot more about someone's software engineering capabilities than short tricky problems.
Some one more seasoned would not work for free during a weekday.
See, I can play that game too.
Besides, this was a contrived problem. If you want to say "I refuse to solve any coding problems while interviewing for a software position", then fair enough. Good luck finding one. I think this was a total of perhaps 3 hours of my time, and was good enough to get an onsite? And it was time I -enjoyed-. Unlike most phone screens, timed interviews, etc.
I think the grandparent is reacting to "Untimed. They gave me a weekend" which made it sound as if the interview took all weekend which a more seasoned programmer would be less likely to do.
If the wrong guy gets hired and everyone is to blame, then no one is to blame. Especially if one of the 8 interviewers has moved on since the hiring, and can be blamed completely, at least on the paperwork. CYA is very popular.
There's also a little manufacturing consent going on. I've been to interviews knowing the guy is already hired because he golfs with my VP, but we'll manufacture some consent by having someone in the group pretend to have input. I can either agree with the VP that his golfing buddy is a great hire, or hit the streets.
Having just had a 3h long technical interview for Google Deepmind, I cannot agree more with a lot of points raised in this post.
Deepmind being a machine learning/statistics/maths/computer science fuelled company, it made sense for the interview process to follow this simple organisation.
I was however very disappointed by the questions asked for each part. Not a single one of the ~100 questions asked during these 3h of my life demanded some "problem solving" skills, only encyclopaedic knowledge (describe this algorithm, what is a Jaccobian matrix, define what an artificial neural network is, what is polymorphism, give examples of classifiers, what are the conditions to apply a t-test...)
So what if someone doesn't remember every definition of the stats/ML/CS/Maths respective bibles as long as they're clever enough to look it up and understand quickly what's needed?
I mean, I get it these are very basic questions but as a highly qualified interviewee who necessarily has other offers given this set of skills, this fastidious, back to school, time wasting process does not reflect well on the company and makes me consider my other options even more seriously.
Do you think mathematicians start over from learning how to count everytime they encounter a new problem? That's different from looking up the 10th digit of pi: that's encyclopedic.
Knowing what a classifier is is simply more than encyclopedic knowledge that you should probably know before joining an AI company.
As I said before, yes these are very basic questions... and I'm not complaining that the questions were too hard or anything, I'm saying that finding someone who will be able to answer 100% of these "definition" questions will not tell you anything about how competent that person is...
Making them face a simple stats/CS/maths/ML problem and see if he/she is able to come up with the relevant concepts is far more interesting.
Ah, I see. Personally I think I could have taken the "give an example of a classifier" question into a deep dive of their competency (talk about how one would build that classifier, etc.) but they might get so many completely unqualified candidates that don't know those things that they need to have those sorts of hurdles before they start really evaluating you for more holistic problem solving traits.
But if you don't think they did that at all then I guess that's bad!
I generally am for the "look things up" argument, but so many people in tech take that to an extreme of "I can fully understand an entire discipline by looking at the Wikipedia page for 5 minutes".
I completely agree with you. Also to be fair towards Deepmind I should mention that this was the first round of interview. Hopefully the following steps will prove more stimulating, in which case 3h for the first step was maybe slightly too much.
My first technical interview they asked me to write a constructor and I completely panicked. It still haunts me to this day that I think the guys in the room thought I was some sort of idiot.
I have this issue as well. I get extremely nervous and forget very basic things during phone and in person interviews.
I like reading these kinds of articles because they make me feel like less of an idiot, but that's not going to help me get over the issue.
I think the next time I'm actively looking for a job I'll have to just buckle down and practice interviewing and whiteboarding code, which unfortunately is time I would rather be spending building real things instead.
I suspect this is the origin of most of the "can't code fizzbuzz" stories. More accurately "can't code fizzbuzz when terrified".
When I was younger I had my best luck at interviews when I didn't give a F and felt I was just showing up for practice because I had a perfectly good job, but this new one looked mildly interesting. I'm not sure reaction to interview stress correlates positively with dedication to your new feudal overlord.
I had a coding test with Amazon to do some pretty basic stuff with a pretty basic data structure. I spent the entire hour panicking, ran out of time, and ended up submitting code that didn't even build. That hurt.
Technical interviews are a glaring example of a problem we don't really recognize: being stereotypically "male technical smart" is a disability.
Having smart people try to determine if another person matches their same definition of "smartness" is fraught with peril. There are a dozen dimensions to "smartness," and not everybody aligns properly.
It's similar to the quote "You have to be twice as smart to fix broken code as you were when you wrote it." Extremely clever code with bugs is provably unfixable. Extremely clever interviews are almost provably bad filter criteria.
But, in the context of interviewing, evaluating _a person_ isn't the same as evaluating _their immediate output_. Evaluating the future output of a person isn't the same as evaluating "can this person replicate intro-to-CS exercises they haven't touched in 12 years?"
Correct interviews require, gasp, strong compassionate people skills in addition to domain knowledge where you can challenge candidates. You've always got to figure out what they actually know versus what they think they know versus what they say they can do versus what they can actually do.
Then there's an entire other issue of "Smart People" versus "Capable People." Most people in power end up being "smart," but not necessarily capable any longer (by "failing upward" and now having magic protections). Some people end up being decision makers with little actual creation responsibilities (read: anything they could actually be judged against), so they are free to just be amazing with little detriment for their decisions. But, sometimes you need a number of "smart but not capable" people to balance out half a company being head-down technically but not necessarily aware of larger issues plaguing them. (Did I just invent managers?)
These essays make me think I don't understand why people are doing technical interviews. I give a (brief) technical question/coding question interview when I'm asked to, and I suspect it's the kind the author doesn't like. But all I'm looking for is:
If candidate lists lots of skills/experience, I do a self-assessment where I get them to claim that they're really good at something, then ask them to do something that requires bare-minimum skills in it. The idea is not that they need these skills to do the job, but that if you claim a CS degree, eight years work experience, and being a python expert, but you can't crack out a basic, non-trick whiteboard coding exercise blindfolded, something smells bad.
If a candidate doesn't have much experience, I'm trying to figure out just how green they are; low-experience candidates cover a huge range of skill all the way down to knowing literally nothing but a few buzzwords. If you don't know a language or a library or a technology I can teach you, but if you don't know anything it's going to be years before I can get anything shippable out of you.
I wish these weren't a major worry but the majority of people who come in the door (after a resume and phone screen!) don't pass.
What do those of you who are doing intensive whiteboard-coding interviews think you're getting out of it?
I've been interviewed by seldo before, when he was at awe.sm. It was a good technical interview. His questions were relevant and forced me to think. I like interviews in which I walk away learning something and I definitely did learn something in this case.
As for interviewing, I cringe when I think about the poor questions I have asked and the poor decisions I have made, and I try to learn from them. We ended up not hiring qualified people because of bad interviews, although that wasn't obvious to us at the time.
I was thinking for a future interview the candidate and I would try to learn some new technology and try to build something together during the interview or talk about it at least.
I'm currently studying for multiple interviews at "established" tech companies, and I wonder where the interviewers like the author are. I've already finished a few, and between having to implement a linked list, analyze a matrix, describe hashmaps and regurgitate sorting algorithms, the questions have not had much to do with everyday coding.
I'd be really happy to discuss how to implement feature X in an app, or how to design the moving parts of Y for a particular infrastructure and talk about the tradeoffs each implies. But I guess I'll get back at implementing a queue using 2 stacks, because that's what technical interviews seem to be for.
This is the most god damn cathartic thread I have read on this site after taking 1.5 years out to work on my own stuff and having failed 4 interviews getting back into the workforce, with the most random god damn shit technical tests.
I see this all the time... people who have CS Degrees, worked as programmers for 5, 6 or 7 years and then they go and start interviewing and flunk over and over again.
I know I have a minority opinion here, but I fell that if a technical interview is such an exceptional event that I have to put that much preparation into it, there is something wrong with the interview process.
Oh how I agree with this post. And I've been the asshole on the hiring end before - but I (hope) have learned my lesson.
I recently interviewed with a consultancy (role was UK-based) that is desperate (quite literally) to hire experienced developers in the technical area I have experience in. I came highly recommended by a recent senior hire of theirs (he had been my manager).
The technical filter question in the face-to-face interview was (paraphrased and simplified): "write an algorithm on the whiteboard to check for mismatched braces".
I described an approximate solution aloud, but did not complete the solution in the interview.
As it happens I solved it in my sleep the following night - but of course they will never, ever know that.
Now everyone will have their pet theories as to why this is a good or bad question, but these are frankly moot because I am confident the hiring developer had no idea what he was testing for with it. He may have a vague "I know the answer, so he should too" mindset, but not much more.
What does this question, presented verbally in an interview really test? A recursive solution sprung to mind, but I can recall implementing recursive solutions in production code only a few tens of times over a ten year career. The question does not test for experience with the technology they are hiring for. It certainly filters out developers who cannot come up with an algorithm while standing at a whiteboard infront of a hostile audience. I could go on, but the biases implicit in this question are as numerous as they are irrelevant.
After a few years in the industry you typically get to work direct with these so-called "top end" consultancies anyway, so you get to know what these kind of questions don't filter for. I can say with confidence and from experience they don't filter for ability to create high quality software.
The thing is, this 'clever' 'does he know recursion?!' question is almost insultingly trivial when compared to the type of problems most development teams face.
Like the fact that although "Joe" has an IQ of 150, he speaks in sentences so impossibly convoluted he may as well be mute. Or "Alice" the sole domain expert, who begrudges her position in some way and will only communicate after you have negotiated the massive chip on her shoulder.
Hell, most organisations can't even provision accurate testing environments and force their developers to run underpowered Windows laptops.
> A recursive solution springs to mind, but I can recall implementing recursive solutions in production code only a few tens of times over a ten year career.
If you are not comfortable with recursion, just remember: Recursion is really just a free stack. To use your example:
Whenever you see an opening brace, push onto the stack. (Or, make a recursive call, which goes onto the call stack...)
Whenever you see a closing brace, pop off the stack. (Or, return from the recursive call, which then pops off the call stack...)
If the stack isn't empty at end of input, then there are mismatched opening braces. If you ever try to pop from an empty stack, then there are mismatched closing braces.
Thank you for the helpful comment. I can't help but feel you miss the point of my post though. I can solve the problem. Verbally, (but admittedly less eloquently and less complete) I approximately gave your answer in the interview. Give me a quiet Monday morning, a pair of headphones and a compiler/interpreter and I can solve it well.
It's the fact that it is being used as a filter when the person filtering doesn't know what is being filtered in and out.
All in all, it's a spectacular waste of time and money.
I understood your post. I didn't comment on your other points because I didn't feel I had anything to add. I was just trying to be helpful in regards to the particular problem you posted, and for anyone having trouble with recursive problems in general.
I disagree with author about coding question. In my experience if somebody is not able to code quickly a simple task (like fizzbuz, reverse a string), it's a terrible sign. It's a simple objective test that filter out a lot of candidates.
Yeah they may be able to be productive in some specific environment (e.g. deep in some framework, writing templates), but in general they likely don't have solid programming foundation and will produce code of lower quality.
That is exactly the problem. Your experience is biased and by definition not transferable to hiring in general. This is why thousands of academic articles exist about research practices and good indicators for future performance, as tokenadult's post nicely highlights.
The thing is, I have seen comments on the fizzbuzz test on blogs. A novice programmer had submitted his solution in the comments and it was almost correct, except he used an "if" instead of an "elseif" or something along those lines. It was wrong, but typing it in a terminal the reason would have been obvious. Its an easy enough thing to trip up on even for an experienced programmer.
This is a really excellent post. It talks about a bunch of things that are hobby-horses of mine (I help run recruiting for a large software security firm), and I find myself agreeing with more of it than I disagree with.
I would go a little further than Laurie does. I think several of the goals he sets up for his process are not in reality achievable in an interview process.
Starting axiom: job interviews are among the most hostile experiences professionals endure in our industry. I think back to my own interviews, and compare them to public speaking, professional disasters, death march projects with insane deadlines, intractable politics, and it's pretty much the interviews alone that increase my heart rate. For the past two years, I've tried to make an effort to peek in on the interviews we do here at Matasano, and what I've seen corroborates the belief. Several of our best hires were physically shaking during their interviews. And we try to be nice! In no other common situation does a tech worker find themselves interrogated by a panel of strangers whose implicit goal is to knock them out of contention.
Given that interviews are a psychologically challenging experience, and thinking about things like the concept of "depletion" of ego or willpower, it's straightforward to see some severe limitations to what can be accomplished in an interview setting. If you're spending lots of energy trying to keep from jumping out of your skin in an unpleasant situation, it's much harder to solve problems that themselves require lots of energy.
Past that, a hypothesis, which is unpleasant for some tech workers to hear (cold comfort: I'm 100% certain it applies to me as well). Software developers are not, as a cohort, particularly effective intuitive psychologists. Virtually none of us have any training in the subject. We tend sharply towards introversion. We train our brains day-in and day-out by repeating tasks that do nothing to developer our ability to read in-person social cues. For that matter, we tend as a group to eschew forms of communication in which tone of voice, body language, and emotional cues are even transmitted!
But several of the objectives Laurie sets out demand exactly that kind of analysis. "Can the candidate intelligently discuss technology?" Well, that's subjective, and worse, vague and abstract. Laurie tries to nail "intelligently" down, but I think we can all see that there are other ways in which someone can be "intelligent" about technology that evade those criteria. Since we all intuitively know that, we substitute our own cognitive biases for "intelligently". All of the sudden, we're gauging "confidence" and "comfort level"... we've decided to be psychologists instead of engineers.
So, two changes I would urgently suggest Laurie consider for his process:
* Audit the whole process for tasks that could generate false positives from a nervous candidate. You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company. Try to build a process that is immune to discomfort and lack of confidence. It can be done! Another thing that we've found very effective: "prime" candidates early and repeatedly with non-adversarial conversations that aim to disarm them. We start our whole (multi-week) process with an hour-long version of this. We also try to innoculate our process by communicating in as much excruciating detail as we can what it will entail.
* Eliminate all subjective questions and standardize what you're left with. Engineers, in my experience, fucking hate this. But it's the right thing to do: ask every candidate, as much as possible, the same set of questions. We have a question structure that minimizes open-ended questions but has some structured "exercise" questions that give the candidate some freedom to move around --- the results of those questions can be put on a scoresheet and, to some extent, compared apples-apples to other candidates.
1. I tried to emphasize how important it is to keep the candidate relaxed and comfortable. Your suggestions are good, though we are too small and resource constrained to spend several weeks on interviews; each candidate gets about 6 hours in total, at least an hour of which is with me.
2. I had this process at my last job -- I asked everybody the same opened-ended architectural design question, and found it very useful for hiring back-end engineers, but at npm it's obviously inappropriate for our front-end and CLI engineers, so I've yet to come up with a "standard" question. It's a great suggestion.
As an aside, I am pleasantly surprised to see how many constructive comments there were on this article. Sort of feels like HN from a couple of years ago.
> In no other common situation does a tech worker find themselves interrogated by a panel of strangers whose implicit goal is to knock them out of contention.
I take the opposite approach. I want you to succeed. I want you to be the best candidate I've seen so far. I want you to be awesome. In fact, I want to hire you right now and never have to do another interview again. I'm in your corner, and I'm rooting for you every step of the way. There's no need to be nervous -- just be yourself.
My explicit goal is to hire someone -- not reject a candidate. I have a high bar, yes, but I start off with the assumption that you can pass it, or I wouldn't have brought you into my office to begin with.
It may be semantics, though. Your implicit understanding is that the glass is half empty, and mine is that the glass is half-full. We're both measuring the glass to see how much is actually there, and if it's not half-full, then that's not good enough. But at the end of the day, I'm on your side.
Having said that, I come from an acting background. I understand how casting works. I'm a tall, dark, lean male that can appear anywhere between 25 and 45 years of age. When I go audition for a role, I'm competing against other tall, dark, lean males that can appear anywhere between 25 and 45 years of age. Depending on the number of other actors auditioning, it may simply be a crap shoot. It may simply be that we were all good in our own way, and they just went with THAT guy because he was the last one in the room, or because he wore that one funny shirt that made him stand out just a bit more.
When you come in for an interview, you're one of dozens of people we've phone screened, and you're one of a half-dozen people that we bring into the office. You've already made it past the first round. Now you're competing with the best of the best that we've talked to. If you don't get the job, it doesn't mean you suck, it simply means that we went with someone else. And if you do get the job, it may not necessarily mean that you're the best -- just that you're one of the best.
Kind of something to keep in mind here.
But again, at the end of the day, I'm rooting for you. I want you on my team, or I wouldn't have brought you into my office. Try to relax a bit, be yourself, and show me that I'm right. After all, I love being right. :)
If rejection wasn't such a powerfully negative psychological force, if loss aversion wasn't such a fundamental part of our psyche, we'd all be successful startup CEOs, because sales would be no problem. But in reality, very few of us have the stomach for sales.
I'm not sure why it makes you sad. On a germane-to-the-parent-comment sense, that's pretty much exactly what tptacek said. But on a wider scale, if you are trying to help someone, wouldn't you want to give them advice that truly helps them in a big and cross-task way? Much better than "get good at Rails", which is useful only for a subset of engineering roles (albeit a subset I may enjoy). In terms of time investment... you can get pretty okay at sales in the time it takes you to be as expert as is potentially relevant in Rails. So why not choose sales?
Yes, for the current state of the industry it is good advice. That is the part that saddens me, that "get good at sales" in order to get a job that has nothing to do with sales is sound, even the best, advice.
Software development is still sales. To successfully roll out a new internal product you need to sell your colleagues on it. To overhaul an existing architecture you need to sell your manager on its necessity. To prevent your coworker from making what you feel to be a poor choice, you need to sell her on your point of view.
Agreed. There is a sense in which almost all meaningful and social tasks are "sales". Scare quotes are used there, not because the same skill sets don't apply, but because you typically aren't actually transacting over the results of the conversation.
> job interviews are among the most hostile experiences professionals endure in our industry
I don't find being questioned during an interview hostile, although I do feel I need to be prepared to be tested.
The most bothersome part for me is companies asking for me to list several references, often specifically asking for previous managers. I am usually given a standard application form to write their names, phone numbers, e-mails etc. before I've even talked to anyone about the position.
An interview is just something I'm involved in. Since I have to hand over contact details of several former managers/coworkers before I even talk to anyone, I have to call up and ask a number of people for the favor of giving me a reference before applying for a job.
Due to this, I almost never apply for one job in a one-off sense. I usually work a few years, then decide I definitely am going to get a new job. I line up my references then apply to jobs until I get one. Then I'm completely off the market for another few years.
I could just not hand over the references until later in the interview process, even though they're requesting them from me before I meet anyone, but that starts things off on the wrong foot. As if I had something to hide.
Of course, there may be legal reasons for companies to ask for everything up front. Asking for references too early is the biggest thing that puts me off from looking around. I'm not going to ask several former managers/co-workers for a favor unless I am seriously looking.
I just don't fill in questions on the application form I don't want to answer. Particularly references. They are unavailable -- since I can only impose upon them so often -- until after all interviews and a salary discussion and a serious decision that I want the job, etc.
There are hostile interviews, but not all interviews are. I'll borrow from Michael Church [1] in describing what I consider a hostile interview. A hostile interview is any interview in which the interviewers are actively trying to remove the candidate from the pool. This can be a very subtle thing; as an interviewer, you have to ask yourself: Am I asking questions in order to attempt to weed out this candidate? Does this question have a right and wrong answer? And if so, what does it mean if they get it wrong? Did I simply discover a limit of their knowledge, or is it more ominous?
An important point is that you can be quite friendly and amenable, while still asking hostile questions intended to directly weed out candidates, as opposed to plumbing the depth and breadth of their knowledge and experiences.
I've had former manager (and friend) call me and ask: "Are you looking for work?" When I told him "no", he told me that a recruiter had called him to "check my references" as a pretense so they could ask him if he was looking to hire anyone.
I would be very careful about giving out references.
> You aren't interviewing people to determine how good they are at interviewing, because interviewing doesn't generate money for your company.
Depends. At most companies, how you handle a simple interview is still relevant if it's for a revenue generating or client facing role (e.g., sales, marketing, and project management).
Granted, most technical roles aren't generating revenue or in contact with clients, so this isn't nearly as important.
Your employers -- or, usually, your employer's bean counters -- may be like many humans in that it's easy to see cash flows and harder to see where value is produced.
This is one of the reasons sales and marketing often have a lot of power in modern business organizations (at least, as long as sales are good).
I've often wondered whether for technical roles involving little or no client interaction[1], negatively-weighting skill at answering very pat interview questions (like where they want to be in five years, what they liked most about their degree course or why they're passionate about writing server side code for an ad platform) relative to their overall performance might actually be a useful, if very counter-intuitive heuristic?
Someone who gives halting, but reasonably acceptable answers throughout an entire interview might be better at understanding technical problems than giving verbal answers in a pressured situation, and find the latter aspect far more difficult than the actual problem-set covered in the technical questions.
On the other hand, someone who confidently, thoroughly and unhesitatingly reels off answers to all manner of non-technical questions should, at least theoretically, be similarly comfortable talking about technical problems they don't find especially hard...
And if they're excelling in the pat interview questions because they've done far more interviews than average than may also be a negative mark
[1]many of which, as others have pointed out, bring in a lot of revenue
I think that falling into the trap of treating social skills and technical ability as a trade-off is a mistake. It's a hackneyed stereotype that I don't think has ever been true--but I'd imagine it has sometimes led to people expressly trying to seem socially abnormal in order to fit the idea of someone technically adept. I recall an anecdote about how a younger mathematician at a top research institution took up running his hand along the wall as he went places in order to seem odder, since it seemed like the best senior mathematicians there all had odd quirks. I'd imagine that the idea that social skills and technical ability are inversely correlated has led to similar things.
But also, my experience in TAing the weed out course for the CS major (and from seeing my fellow TAs teach and work, who were more or less the best in class at my University--so much so that I was flown out by a company simply because of my involvement in that class) was that social skills had absolutely no correlation to technical ability: your ability to understand how pointers or complex data structures worked had absolutely no relation to how socially fluent you were. In fact, it often went the opposite way, up to a point: there was some base line of social fluency that everyone I worked with and the best people I taught seemed to meet. The single best coder I have /ever/ seen work (he alone makes me absolutely certain that the best people in our industry are 10x/100x more productive than the average) was also one of the most comfortable with people and had a very clear idea of his career goals. I'd expect him to give confident, unhesitating responses to "Where do you see yourself in 5 years?" and "What's your greatest strength/weakness?" Negatively weighting that would mean you'd miss the hands down most productive person I've ever seen work: and you'd probably miss a decent number of the other people I taught with. (Who, as I said, were basically the cream of the crop from the program I was in. Not all of the best people were there, but all of the people there were among the best.)
Now, I don't think they're particularly good interview questions, even though I understand why they're asked. I think that asking questions about actual workplace soft skills (conflict resolution, for instance) would be a much better use of everyone's time, since in the end it's those things that actually hold a team together.
It's less to do with social skills (agreed, there's no trade-off) and more to do with the probability that people scoring 6 out of 10 for their ability to discuss technical problems whilst getting 10 out of 10 for their ability to answer general interview questions are good interviewers with a weakness for understanding technical problems, whereas those scoring 6 out 10 for technical problems questions but 5 out of 10 for generic interview questions are likely underrepresenting their technical understanding due to the awkwardness of the interview situation. I'm also assuming that some common non-technical questions (like biggest weaknesses, and what they actually did in their last job) usually elicit useful information as opposed to answers synthesised from the job spec.
Since I've introduced arbitrary numerical scoring, I guess by "negatively weighting" I actually mean doing something equivalent to mutiplying their overall performance by 1 + scalefactor*(technical answers score - generic interview answers score) , but I'm assuming in practice most interviewers use their intuition rather than subjectively scoring every answer.
I'm assuming your amazingly productive friend aces the technical questions at least as impressively as the "convince me you'll have grown with the company after 5 years, even if you actually only see this as the bottom rung of the ladder" ones.
I think you miss the point. It's not that people with social skills are likely to be worse technically, it's that social skills help you even on the technical questions in a setting where you are basically trying to sell yourself and being judged socially. Weighting the purely-social questions negatively might serve as a control.
A reasonable theory; I'm skeptical as to whether it would turn into better performance in hired candidates.
I think that you're right about this. People with poor social skills are going to be worse at negotiating pay and in much lower demand, overall. On the other hand, I'd assert that they're going to be harder to manage and lead to more interpersonal issues (almost by definition).
>> Somebody who can intelligently discuss technology
>> Somebody who knows what they don't know
I believe in these principles especially. Most of my interview questions are vague (and I tell the candidate this up-front, and explain why). For instance, I'll ask them to explain how they would debug a very slow cluster, or to explain everything happens between me hitting the keys 'google.com' to me viewing the web page. This gives them a huge range of topics to cover in as much detail as they want. If they know a lot about Linux administration, or how hardware interfaces with the OS, or a lot about network protocols, they have a chance to show it off. I'll drill deeper into wherever they take the conversation, and a person scores major points with me if I drill down far enough that they get lost and say "I don't know, but my guess would be... and I would confirm that by...". I've found it to work exceptionally well.
Oh god, a question like the Google one would have me not knowing what to say... I mean, to give a reasonably complete answer would take hours and hours! What do you do if someone really goes into detail and can't even get to the part where the packet leaves the local network in a practical amount of time? Is that viewed as inappropriate overkill or a poor understanding of the question?
I wouldn't fault anyone for being able to go into so much breadth and depth that we ran out of time. I would suggest they just dive into the part they think if most interesting, or the part most relevant to the position. When we're out of time, we're out of time, but the point is just to stimulate enough technical conversation for me to evaluate the candidates knowledge, passion, communication skills, strengths, interests, etc.
edit: For instance, I actually got this question asked to me in an interview and adopted the practice several years later. It was scheduled for an hour, they asked me this immediately, and I was talking for most of the hour. I got through the whole process but didn't go into as much detail as I could've along the way. Several times I was stopped and asked to clarify some details, some of which I didn't know. That was that, and the interviewer just said "very good" and left. Looking back, I think he was doing the exact same thing I do now - giving me the freedom to show off what I knew, seeing if I would be honest when I reached my limit, and probing a bit to see if I could communicate about interesting problems.
I think you're allowed to ask the interviewer how detailed they want to be. I could spend an hour explaining how keyboard drivers work, if you want, or we can just assume that's a solved problem and talk about TCP, DNS, and SSL.
I read the comments already posted in this thread before I read the fine blog post. Almost everybody sucks at interviewing. Research shows that even though job applicants think that an interview is one of the more fair procedures for hiring a new worker for almost any kind of a job, it is one of the least effective.
There are many discussions here on HN about company hiring procedures. Company hiring procedures and their effectiveness is a heavily researched topic in industrial and organizational psychology, but most hiring managers and most job applicants haven't looked up much of the research. After reading the blog post kindly submitted here, I'll make some comments on its tl;dr summary at the end of the post.
1. many interview techniques test skills that are at best irrelevant to real working life
Yep, that's why you want your company hiring procedures to be based on research and what really matters for finding good workers.
2. you want somebody who knows enough to do the job right now
That's the ideal. That's why work-sample tests are, by replicated research, a very good hiring procedure, one of the best possible hiring procedures.
3. or somebody smart and motivated enough that they can learn the job quickly
Yep, and that's why tests of "general mental ability" are also a very effective hiring procedure, although there are some legal requirements surrounding use of those that you have to be careful about in the United States.
4. you want somebody who keeps getting better at what they do
For sure, as that is the only way your company can meet new challenges as they come up in the company's business.
5. your interview should be a collaborative conversations, not a combative interrogation
I'm not sure that the author here has provided evidence for the "should" statement in this summary, although I actually don't disagree as a matter of how I do job interviews.
6. you also want somebody who you will enjoy working with
Basically, almost all hiring managers fall victim to overemphasizing likability and underemphasizing ability to get the job done, but, yeah, you don't want to hire someone who makes the workplace miserable--that might cost you losing other good workers.
7. it's important to separate "enjoy working with" from "enjoy hanging out with"
Absolutely. The best worker in your company may not be the same person you go out with socially after work.
8. don't hire assholes, no matter how good they are
The trick here is to figure out how much annoying behavior qualifies a person as an "asshole" in a particular context, and that is not easy.
9. if your team isn't diverse, your team is worse than it needed to be
There is an increasing body of research to back up this idea.
10. accept that hiring takes a really long time and is really, really hard
Hiring is hard. It may or may not be time-consuming, depending on how efficiently you do it.
The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,"[1] Psychological Bulletin, Vol. 124, No. 2, 262-274 sums up, current to 1998, a meta-analysis of much of the huge peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.[2]
EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, with its legal rules about hiring, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case[3] interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. A company defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)
The social background to the legal environment in the United States is explained in various books about hiring procedures,[4] and some of the social background appears to be changing in the most recent few decades, with the prospect for further changes.[5]
Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures"[6] (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:
"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."
But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."
Thanks for posting this - some real food for thought here. Can you explain exactly what you mean by a "work-sample test", in particular how that might differ from a pair programming or take-home coding exercise?
Can you explain exactly what you mean by a "work-sample test", in particular how that might differ from a pair programming or take-home coding exercise?
This HN search for comments by Thomas Ptacek on the topic of work-sample tests
will lead to better descriptions of work-sample tests useful in a Hacker News context than anything I have yet written on the topic. I'm grateful to him for contextualizing that issue for this community.
I think there are some good points here, but I would argue that a lot of these are essentially points for hiring more junior engineers. There are cases where hiring for what someone already knows DOES matter.
I have come across too many codebases that were clearly written by folks who were in the "get stuff done" mentality. Frankly, when you're learning a new technology in addition to trying to do your job, your code/architecture just isn't going to be that good off the bat. If the company had instead hired someone who knew what they were doing from the get go, they could have started with a more solid foundation and potentially avoided pain down the road.
IMO this is a red herring. Even ignoring shifts in language and such, I'd imagine/hope that just about everyone would cringe a bit if they looked back at code they wrote some time in the past. It's natural - we're constantly learning and constantly improving.
I've changed languages multiple times in my career and yeah sure, one isn't writing idiomatic code right away but with a little breadth of knowledge and armed with modern tools such as stackoverflow it's really not a long transition period. If you add in a strong team and a strong culture of code reviews, the really egregiously non-idiomatic code will never make it in the codebase anyways (not to mention, the newbie will learn along the way).
Is that a problem of not knowing the language or a problem of not having exposure to better patterns [1] that are generally language-agnostic? I get what you mean about the idioms of some language: code is easier to maintain when it looks like what others on the team are accustomed to. But if that's the problem we're trying to solve, isn't that what initial code reviews of new team members can help resolve, among other things? And isn't that a benefit to hiring people who know other languages well? that they bring in new perspectives and ideas, such that the whole team can grow, over time learning and incorporating those new patterns?
Personally, I look for people who can think, who have exposure to a variety of patterns and can synthesize those patterns into their work, making appropriate choices. More power to us if, for our Perl shop, we hire someone with a strong background in modern Fortran because that different perspective is part of what makes the team diverse, creative, and successful.
I don't look for an expert in Perl to be even a senior developer here. It helps, but the primary consideration is their ability to work with abstract information. The language is just one of many tools to express that solution, and the language choice can change over time to suit needs.
[1] Not to be confused with "design patterns". I am using the term more fundamentally.
No really. I was hired in my current job 3 years ago, with experience. I didn't know Django, but had read enough, and seen a talk on it by someone else. I learned Django on the job, but my previous knowledge of databases and Perl complemented that. I didn't know Python, but its similar enough to Perl, I just googled to find the Python equivalent. Once you have a seen a couple of languages and concepts, learning new ones doesn't always hinder you so much.
One of the things done right at one place where I did a great many interviews for technical candidates was at a late phase to break out the technical interview and the management interview.
This allowed a group of potential peers to have a friendly discussion of varying depths about not only the recent work and advertised skills of the candidate but about current events and community engagement.
We became adept at evaluating candidates based on how they answered problems we presented, whether we believed they actually had the skills and experience on their CV, and how committed and engaged they were to the profession.
This is especially important for security related careers because community engagement is vital, most people can't talk in detail about previous work and may not have been able to publish public material, and work is highly specialized. It is tough to evaluate someone on skills and knowledge they have and you don't, but we got good at it.
And then the managers could talk to the candidate in a separate panel and ask ... well I have no idea to this day, actually :)
I say more about all of this a presentation I did in February, Breaking Into Security: some InfoSec Career tips, presented at DC404, slides here:
http://www.atlbbs.com/sharkin/breakin-dc404.pdf
Instead of technical interviews, I would love to just pay a potentially great candidate to come in and work for the day. Not only would it give them an opportunity to demonstrate what they know and show off their other skills, it would give our team a chance to see if they're a good fit. And, there would be some real financial incentive for them to give us a try.
But, trying to get management and HR to change from the conventions is extremely difficult. I have not been successful.
There are big problems with processes that work this way --- as a group, they're "temp-to-perm" arrangements, although your "one day" version of it occupies the less odious end of the spectrum. I've written about the idea elsewhere on HN before and will try to dig some links up.
The biggest problem with a "work there for a day" interview IMO is that there's no time to do anything useful. You can't get familiar with a codebase of any size in a day.
Welcome to pair programming. Unlike real pair programming, don't let the noob touch the keyboard for the entire day. Talk talk talk. Don't expect either to really get much work done. But pretending to work all day is a heck of an interview.
This works outside of programming. Over a decade ago when I was doing network stuff, I did "pair router work" with a couple people. You don't give a noob enable CLI on a production router on his first day, or during an interview, either. Even if on paper, or in reality, he is better at this than you are.
If you hire them, you'll be spending the first couple days like this, anyway, unless you go all sink or swim.
good point, especially since all day interviews are most likely requiring the candidate to take the day off anyways. On the other hand, i'm a mess the first month starting a new gig with all the lingo that I don't know yet...
Followup question: Where can I find a list of companies (other than npm and Matasano) that don't interview with the stupid "write code on the whiteboard" method?
I've got some (1.5 years) experience, and I can write code. But put a whiteboard marker in my hand and ask me questions you know the answers to, in a high-stakes, high-pressure environment, and I fall apart.
Case in point: a recent technical interview I did, I did reasonably well until the last 1 hour session, which was the "code on the whiteboard" problem. I got code that functioned but wasn't fast enough. At home, after 1 google search and a few minutes, I solved the problem.
I'm definitely in favor of the "do a small project" type of code interview. It has its own pitfalls, but it's way better, IMO.
While I can see potential danger in hiring friends, I think it's wrong to say outright that it's a bad idea.
Most bad hiring of friends comes from being bad at hiring in general. If you hire someone BECAUSE they're your friend, then you're doing it wrong. The idea is to hire people who are qualified and who you know to be quality over a long period of time.
Used properly, friendship can just be considered a source for good references in this case, and potentially some additional glue during tough times.
It's true there are risks to hiring friends, especially if it's done poorly or for the wrong reasons, but it's also true that there can be extraordinary benefits.
My go-to question for technical interviews is "how would you write Scrabble?". It's very open-ended. I expect them to ask me questions like "what do you mean, write Scrabble?", to draw out some class diagrams, and to implement some scoring. It shows how self-directed they are and how much they trust themselves to develop architecture.
I hate questions about bit-twiddling and sorting algorithms. I wouldn't remember that stuff because I never do it. I do expect candidates to be able to analyze an algorithm for time and space requirements, because that's important for pretty much any coding job.
I expect them to ask me questions like "what do you mean, write Scrabble?", to draw out some class diagrams, and to implement some scoring. It shows how self-directed they are and how much they trust themselves to develop architecture.
This is a psychological test. You describe how you would imagine yourself solving it -- which may or may not be accurate -- and think that's how "good developers" would approach the problem.
The candidate doesn't know you. You might be the person who says "I'm looking for the candidate to tell me that is a stupid question" or "I expect the candidate to make a first simple iteration planning to throw it away instead of bothering with UML diagrams."
For the candidate, this is a game of rock-paper-scissors where they have to guess the state of the PRNG sitting across the table from them. There is a much better route: tell the candidate what you want[1] and see if they give it.
[1] Tell them directly; it shouldn't be buried in a post on your company's blog. I have encountered this pattern myself.
I would fail this. I know of Scrabble, but I have not played it. On top of that, from having had to build scoring for things previously, I would say that it's non-trivial, which would compound the fact that I don't know what goes on on a scrabble board
If you didn't know Scrabble I'd ask you to code another game. The fact that it's non-trivial is kind of the point; you have to ask questions (gather requirements) to do it.
I've interviewed with dozens and dozens of startups in NYC. I'm an entry level Front-End Dev with passion for a lot of things I (in hindsight) probably should have gone to college for. For example, my current side-project is a completely client-side image metadata reader/writer that has forced me to learn new things like parsing Binary, Endianness, Meta and a lot of things about organizing JS because I've never built this large of a project completely by myself even though it is very, very small. A weekend project for most of you.
Throughout all of my interviews I've noticed one common trait.
Either I do exceptionally well on the personal questions during the interview, or I do really well on the technical questions. There is no in-between.
Most of my interviews could be broken up into 2 sections. The first section is usually where they open up and try to make things more comfortable. Asking me where I'm from, how long I've been writing code, what kind of side-projects I'm working on, etc...
The second part of the interview is when they try to figure out how much I know. Usually we start off by just talking about technology and this is where they try to figure out if I'm BS'ing my way through things. If you can maintain a fluent conversation about technology; you're good-to-go. After that they will generally try to slip in a few questions, usually about event delegation in Javascript.
The problem for me comes from when we switch from personal questions to technical ones, or vice-versa. I will pass either one with flying colors but I will rarely, if ever, pass both. If that particular day I can't sell my personal story very well, I do very well on the technical questions. If I do well on the personal questions, I do terrible on the technical questions. I believe there is a vast cognitive gap when you go from an emotional conversation talking about significant things in your past (like side-projects, old employers, hometown, etc..) and then jump to such rigid topics like code where there is no emotion, it's purely analytical thinking.
What I would prefer is for a company to pay me $100 to come into the office and work a half day. If I can keep up, I get the job.
Not one company has invited you to come onsite for a day-long interview to code alongside the team?
You should be upfront with the employer. "I think the best way for me to showcase my skills would be to come onsite for a day, and tackle whatever projects you're working on currently. After 8 hours, you'll have a good assessment as to whether or not it's a fit, and vice-versa."
Not one company has asked me to write code on a computer.
Also important to note these are almost all fully funded companies with solid dev teams. I've had great offers that just never worked out due to fit (or rather lack of passion from the dev teams).
I've thrown the offer out there on several occasions but it's always refused or delayed to the point where contact drops. One time the senior engineer told me my offer to come in and code on my own dime was "nonsense" and If I knew better I wouldnt be offering my work for free, because that would hurt the "startup economy".
My faith in the NY Tech Scene is pretty much non-existent to the point I'd rather make hourly wage in manual labor and hack on projects in my own time, as opposed to burning myself out around people like noted above.
> I used to ask people to write code in interviews. This is terrible.
Bull! Last time I interviewed someone, they looked good on paper and decent on the phone. When it got down to solving a _simple_ problem on the whiteboard, he totally flopped. This is a totally realistic situation; we get together at least weekly and hammer out a solution on the whiteboard. Nerves could be an issue, but a good candidate should be able to solve an easy problem in a handful of minutes in an interview. (I'm talking _super_ easy, like joining one SQL table to another after saying that you are proficient)
He didn't say coding was bad. He said coding on a whiteboard is bad. And, if you evaluated 100 strong developers by comparing their performance at a terminal and a whiteboard, my guess is that you'd quickly see that he's absolutely right about that --- not only would everyone outperform the whiteboard with a terminal, but that the whiteboard deficit would be highly irregular, defying any attempt to "handicap" for it.
Coding on a whiteboard is also impossible or infeasible for candidates with some kinds of disabilities (e.g. blind). Sure, you can make an exception for those few candidates, but I suppose the fact that you even have to make an exception to your normal process for a particular candidate might bias you against that candidate.
There are lots of accommodations I would need to do for someone who is blind, or deaf, or missing his hands. (I knew (through association) a developer who had hooks for hands.) The fact that you need to put some changes into your recruitment process for disabilities doesn't mean your recruitment process is necessarily broken.
You use Emacs without autocomplete? That's not snark: I'm just here to tell you that Emacs has fantastic autocomplete and you should definitely look into it.
Being able to memorize things is valuable to a developer. I'm with you that other things are more valuable, but you don't always have to make a trade-off.
This is one of the best write-ups over tech hiring I've seen in a while. For me is about "get the job done, don't be a dick", problem is how to test for that under time and other constraints.
Not sure if this is off-topic but my concern is related to technical interviews.
How do software engineers/devs/etc "stay in shape" for technical interviews? Let's say I wanna switch jobs and the first hurdle to cross is inevitably algorithms and data structures (for dev-centric roles) or something like operating systems/networks/whatever. How does one stay sharp on those _fundamental_ skills on a regular basis while working on a job, to avoid cramming everything in one week before the interview?
The other side to that story: I'm scared like shit whenever (and that's often enough) I consider applying for a new thing. And back out, don't even try.
Most of the reasons are listed in the article, but let's be honest: I'd face most of these things.. So the pipe dream atm is to find some time for personal projects, foss contributions and .. use that to get to more reasonable opportunities. Reaching ten years with this company here, these interview stories might keep me here for another decade.
If it helps: when I interview, I really really want the candidate to succeed because he or she is solving a problem for me. I need more competent hands. I don't know if this helps, but it's true.
I'm not sure I agree with the "team fit" thing. I get his point - most people don't understand what team fit is or can't separate it from personal bias. But here's my counter-argument.
Every company and team has different core values. "Team fit" means matching company and team values. For example, I work on educational software for teacher and students. My definition of "team fit" (for this particular team/company) is that the person care about improving quality of education. If you're applying just because you need a job but don't care about what it is we're making, I don't care about any of the other criteria listed in this article - I'm not hiring you. You may be smart, able to learn, and a great communicator - but you don't care about what we're building so you don't have the same core values as the rest of the company and team.
This is what "team fit" means - not what they look like, what they do in their off time, or anything else; "team fit" is "do they personally value the same things we value as a team"?
If you value transparency and collaboration and they prefer to work as a lone gunman, they're not a team fit. If you value rapid iteration and user feedback and they don't want to release anything until it's 100% perfect in their eyes, they're not a team fit. So on and so forth with the examples.
So don't call it "team fit" or any such vague terms. The problem with these generalities is that they can be (and are often) used to hide personal biases and discrimination.
You work on educational software for teachers and students, so evaluate your candidates based on clear, specific complaints. It's not "bad team fit", it's "this candidate doesn't care about education software".
Eliminate these vague generalities in favor of specifics. This is useful for yourself and your team, since people are forced to be more introspective when they must be articulate about why they dislike someone.
"Bad fit" is a cop-out that allows people to dismiss others without having to issue specific complaints that might cause their brain to go "wow, these words make me sound like a dick".
Most people are fully able to work in both 100% done and release flawed but soon scenarios. Most people simply adjust to company they work in just fine. Almost all are also can work in both team and lone gunman settings - although they will have preference there.
Unless you are very careful, you can select out very good candidates basically based on how companies they previously worked in performed in those scenarios.
There's a difference between what you CAN do and what you VALUE doing. Yes, people can put up with conditions they find undesirable, but they'll be unhappy and cause others working with them to be unhappy as well.
> Variation across teams in big companies is enormous. Just because a company was successful doesn't mean your candidate had anything to do with that.
This might seem to be an over generalisation. It's probably true enough at large consulting body-shops but I'd certainly punt on a lower-end ex google dev comparing favourably to the industry average.
All of these exact same points (including a very similar title) have been floating around my head as an idea for an article I wanted to write. I was so pleased to find that you had basically written my article for me. Very well said on all accounts :D
As someone who recently graduated from college, by far the best interview I ever had was just question all day and asked about projects I would want to work on and asked to explain how I would go about designing my project on the whiteboard. No I didn't get that job; I screwed up the last part (well I did answer some questions incorrectly but I think they didn't care much about that). For what it is worth, the position has to do with security research and engineering. The coding part I had to go through was fairly quickly (but the interviewer, IMO, didn't do a good job at it and caused me a lot of trouble). Geesh, I wish I was given a offer, but looking back I wasn't ready to deliver ANYTHING valuable as an [researcher]engineer yet.
When people just say a bunch of smart words like async and parallelism, ask them to define them with concrete examples. Ask them further to break down their "textbook knowledge." That's by far, IMHO, the best way to evaluate whether a junior prospective engineer knew a lot than average or not. You want a hard question? Just ask people to go as deep as they can to explain to you what happen after a URL is entered into a HTTP client such as browser or curl.
I am terrible at coding under pressure. Actually, I am terrible at coding. I constantly read books and github projects and I end up asking a lot of questions: is this a good practice to do such. You see, I am all for style and I want to be better writing code close to what people prefer to write. I went to talks and learned tips how to write better code and I will immediately employe those new techniques in my new projects.
I will end up with questions and I always wish someone could be around and help. I usually go to IRC or stackoverflow for opinion but I hope when I do get a job soon, I will be able to ask my senior colleague and hope they could provide feedback.
I haven't done everything like some superstar already have at my age. I suck and I am sorry, but I think I am one smart ass person capable of delivering a project with some guidance and day-to-day meeting and mentorship.
In fact there are senior engineers I've met or worked very very briefly during hackathon and when I asked them certain design question they will blur about their ideas and don't really know how to contribute, or up to me to explore what could be done. In essence, neither of us know everything. Senior knows more because they have done more, almost certainly something repetitive. It is like asking me to write hello world every day and I will have no problem responding to it. Sometimes when you look at code written by senior engineers their code smell even worse than what I write (but there are definitely the good one I can always use as reference).
Conclusion is: when you want to hire a new engineer, especially a junior engineer, ask yourself: are you ready to be a mentor and is there anyone ready to believe they can be a good mentor. Actually, when you hire someone, ask your co-workers whether they are ready to be each other's mentor. I don't want to work at a company where everyone is hiding in a cave and think everyone is smart and autonomous. There are times you need to be there to guide someone through his or her obstacle. A company encourages brownbag and training will be ideal for me.
I am serious; don't ask me to code quick sort. I am not going to do it. I know I am just going to memorize it and after I've written one a dozen time it will just become hello world. No, I am kidding. I need a job so I will do it. I will, but feeling a time bomb ticking next to me. I will either fail or go on with another few rounds writing more quick sort.
I think a more apt title is "technical interviews suck", and speaking personally, I have entirely given up on them. (I would say that I gave up on them after two decades, but the truth is that I gave up on them a decade ago -- and I really tried hard in that first decade to develop the perfect technical interview.)
My belief has become that the only way to hire the traits that I'm looking for (high technical ability, sufficient education, predisposition to rigor, and -- most importantly -- indefatigable persistence) is by judging a candidate on their work, not their performance in an interview. (After all, software engineering isn't done via pop quiz -- and it's not even a timed test.)
The problem then becomes: how do you judge the works of someone you've never worked with before? Three ways that have worked for me:
(1) Rolodex. This is an easy way to hire reliably: someone on the team has worked with the person in the past, and vouches for them as one of the tribe. Assuming that you trust the person vouching for them, the interview consists of you selling them, not them selling you. This method has high fidelity (though is still fallible), but also suffers from obvious limitations in terms of scale and breadth.
(2) Known curricula. There are some schools where I know (or someone on the team knows) the computer science curriculum very well, and can assess a student simply by the courses that they have taken (or, better, TA'd), the labs they have worked in, or the professors that they have worked for. The fidelity of this method will depend on the school, how well one knows the curriculum, etc. -- and it has all of the strengths and weaknesses of hiring university grads. (It also suffers from serious scale problems and runs the risk of creating monoculture.)
(3) Open source. If you lead open source projects that attract community contributors, you may find it surprisingly easy to coax the best of those contributors into working for you full-time. While I know that this isn't a fit for every company, it has become my preferred way to hire: you are hiring someone who can obviously do the work (because, um, they're already doing it) and who has demonstrated interested in the problem space (especially if their contributions have come during their free time). Importantly, you are also not limiting yourself to a particular geographic area or school or company history or whatever; the people I have hired via open source are people I never would have found otherwise. And, it must be said, they have proven (without exception, actually) to be great hires. There are obvious problems with this as well in terms of scale and (especially) predictability, but I view open source as the farm system for software engineering: it's a way to find and develop talent at lowest opportunity cost.
Edit: I forgot a fourth method that has actually worked for me.
(4) Homework. When interviewing someone who I don't know and who is otherwise a stranger, I will ask them some exceedingly basic questions to assess technical viability, and then proceed to discuss the problems that we're solving and why I personally find them exciting. If they are sufficiently interested at the end of this conversation (which is really more me selling them than them selling me), I will assign homework -- which is generally to take some fixed amount of time (I have usually said no more than eight total hours) to build something fun in one of the technologies that we have built or otherwise use. (When node.js was new, this was to build something in node.js -- but more recently, it's been to build something fun with Manta.[1]) If a candidate comes back with completed homework, you each know something about the other: they were sufficiently interested in you to actually play around (and they like playing around to begin with -- which is essential for us) and you now have some works by which to judge them.
Homework is great, but only if a job offer is guaranteed if submitting a sufficient project. It is hubris to think someone will care enough about your company to spend 8 hours of their time for the small chance to be hired. Presumably you are giving the same assignment to a handful of other candidates. You are simply outsourcing much of the investment of hiring to the candidates. This is simply unethical (one of many tests for ethics is "if everyone did X, would the scenario be tenable"--in this case the answer is obviously no).
It's not a small chance at all -- and it only comes after a conversation where I have done basic vetting and where I have explained why the problems that we're solving are exciting to me. (That is, if you're not ginned up to do the homework, I haven't done my job -- or it's not a fit.)
And as an aside, your test for ethics is entirely asinine, as it renders anything of sub-infinite scale unethical. (Among its more obvious failings, anyone who chooses not to have children is behaving unethically -- as is anyone who has children, for that matter.)
Homework is going to put a bunch of candidates off unless its a short amount of time (~5 hrs) and as an alternative to taking time off of work to go do an in person interview.
I know one guy who did 30 hours total of interview investment, where a bunch of it was homework and was eventually not hired because he wasn't 'passionate' enough about payroll software! You don't want to waste your time doing such shit so you quickly start refusing long homework projects. Your in demand and you have better things to do with your time.
And as an aside, your test for ethics is entirely asinine
To me, "if everyone did X, would the scenario be tenable" sounds like a less fancy way of saying "Act only according to that maxim whereby you can, at the same time, will that it should become a universal law."
That is to say, Kant's Categorical Imperative [1].
If you consider it asinine, I could introduce you to a lot of intelligent people you would think were asinine.
Of course small is subjective, but I would consider, say, a 1 in 3 chance to be small in this case.
>test for ethics is entirely asinine, as it renders anything of sub-infinite scale unethical
I don't see how this is so. Explain?
>anyone who chooses not to have children is behaving unethically -- as is anyone who has children, for that matter
This is also not obvious. Of course tenable is poorly defined, but there is nothing contradictory/chaotic/unsustainable about everyone stopping having kids. Furthermore, it is not obvious to me that the statement "it is unethical to not have kids (if you're able)" is necessarily absurd.
> The famous fizzbuzz test simply asks "are you aware of the modulo operator?
No it doesn't. It could be implemented with counters you reset when needed. Didn't really read any further, I don't think this person should really be giving technical interviews anyway.
I thought the point of the FizzBuzz was that the asker doesn't care about 'clever' (for some definition of clever) answers, but rather, "Can this person actually write something which solves the problem, no matter how brain dead a solution"
Only if you have no workaround or ability to implement modulo operations. If you were really a capable coder, but didn't know of %, you would just say, "I don't know of a library function for finding the remainder, so I'll just plug in remainder(x, 3) here." Then later, if such completeness is desired, implement that function.
1) Do you have basic problem solving skills?
2) Can you communicate clearly?
3) Do I want to sit next to you for the next 6 months or longer?
If you don't know Ruby, I can teach you. If you don't know Elasticsearch, I can teach you. What I can't and don't have time to teach you is how to solve a problem on your own without me holding your hand, and I especially don't have time to waste trying to communicate poorly with you.
And obviously, I want to work with someone who is pleasant and interesting. I don't wanna sit next to someone for 40 hours a week who stinks or is rude or can't carry a conversation.
If you meet those three criteria, you've beaten 90% of candidates that walk through the door. The last 10% is what gets you the job. (Obviously, if you DO know Ruby or Elasticsearch, that's a huge plus... but it's not one of the bare minimum requirements.)