The modern FAANG Frankenstein interview is a mess. It was so/so at Google 15-20 years ago, it was so/so when initially FB but most everyone cargo-culted it 10-15 years ago. It’s become “grind leetcode” which is clearly a failure mode.
The trouble is it’s a hard problem, and it usually gives -some signal, so it’s sort of better than nothing? I guess? In cases where contract-to-hire make sense for both the company and candidate I generally regard that as ideal, but that’s not every situation.
Someone will solve this, and that person will be very well-loved.
Google did the "how many gas stations in the us" interviews for years before finally realizing that doing well on questions like this had no correlation to success on the job. Now, in reaction to that failure, everyone is doing the LeetCode thing instead, and presumably will eventually realize that this doesn't correlate either (especially irrelevant in this CoPilot era when ability to memorize and code up an algorithm is becoming about as relevant as ability to drive a stick-shift car).
There's really no substitute for interviewing based on candidate's experience.
Apparently the new job market trend is applicants firing off dozens/hundreds of AI generated applications, which are then screened by an AI on the other end.
Haha I’m not sure I agree about how useful CoPilot is in practice (there are certain tasks where it shines but all of the LLMs print invalid code routinely).
With that said you made me laugh out loud because I have a friend who knew two people trying to LLM some minor contract and it just went on forever. I definitely think two LLMs talking to each other with two people copy pasting and not realizing it is worth like, a Rick and Morty episode or something.
Yeah, I've even seen people in top ML jobs glowingly describe the main value of CoPilot as (just) smart autocomplete, but copying/regurgitation of algorithms ought to be something that even today's LLMs should be capable of.
Of course there's always Google search too if you are just looking for an algorithm, but LLMs help in the discovery process since you can describe what you need without knowing the name of it.
Using LLMs to find reasonable subset of algorithms for a problem sounds like a valid use case. Could even get naive implementation and lead to comparing them to pick right one.
If they take the LLMs advice were they astute or just lucky? You'd have to ask the candidate to go into details to get any signal, and if the algorithm is hard enough for them to have to consult an LLM, they might not be able to do that.
I was referring to LLM use on the job (or hobby), not as any part of interviewing process.
The reality is that most programmers can go an entire career without being algorithmically challenged, and on the rare occasion you need to do/optimize something and don't know what the best options are, then you can either just ask a colleague, or ask online, or Google, or nowadays discuss it with an AI!
Why do you think they haven't adjusted again? Is it possible it is actually (loosely) correlated to job performance?
I interviewed at Google and all the questions were practical, challenging, and not found on leetcode.
I've seen all kinds of interviews in my twenty years of experience, and while all types can be done poorly, (including DS&A ones), having some live coding is one of the best signals you can get in a short amount of time.
I've been blown away by a candidate while they're talking about their experience, but then asking them to code something small, they utterly bomb.
Conversely, I've seen shy, humble candidates struggle to articulate their skills, and then crush harder and harder coding challenges.
> I've been blown away by a candidate while they're talking about their experience, but then asking them to code something small, they utterly bomb.
You're making the opposite point you think you are. How do you know that you observed an inability to code as opposed to interview performance anxiety?
I've been coding for 25 years, but have bombed my share of very simple white-boarding exercises because I panic and my brain absolutely stops working. If it's experience-based or even take home, I generally knock it out of the park.
White-board coding works for FAANG because they have 100,000 applicants at any given time and it doesn't matter if it has an exceptionally high false negative rate. It does not work so well for shops that are struggling to find candidates.
>You're making the opposite point you think you are. How do you know that you observed an inability to code as opposed to interview performance anxiety?
I've done lots of interviews, and I know that if someone has seen a question recently, or if someone is rusty in interviews, or one of a million things can affect someone's live coding ability.
So I'm not looking for speed, I don't give more 'points' to someone who is calm and collected vs. someone who takes twice as long cause they're nervous. They both "pass" in my book.
I also don't particularly care if someone starts intermixing java and C++ syntax or forgets the string library and says len(var) instead of var.len() for example.
I'm even fine with giving hints. But there are literally people, with hints, who cannot write a basic recursive function or understand to check for null references. That isn't being nervous. That's just...being bad at the craft.
It absolutely does not always happen. I was asked to do interviews twice with one employer. Both times, I requested the interviewee's resume. Both times I was refused because they didn't want me to "bias" my interview.
It’s not generally the responsibility of the person doing the technical screen to also obtain signal on past experience. At a company with recruiting process these fall on different people specialized in the specific interview types. I’ve done an average of 2-3 interviews a week over several employers for the last 12 years, so over a thousand interviews. I’ve also defined much tech interview loop for my current employer and usually interview from junior IC up to manager of manager roles.
I’ve never asked for candidate resumes because I don’t want to bias myself.
I’m not saying it will always, always happen, it’s just, it overwhelmingly will.
If you don't have a resume, do you still ask about candidates experience?
Many people can talk a good game and come across well but still be useless (e.g. hard fail on the simplest of white board tests).
I haven't done interviewing (on company's side of table) for a long time, but my main tack was always to go over projects on resume, focusing on last 5 years, and drill down to see if they could fluently talk about them, draw architecture diagrams, justify choices, etc, etc.
I you don't ask about past experience (= same as having resume), then how do you even know what the person's experience consists of? Useless cog, or tech lead? Highly agentic or passive, etc?
Maybe we were talking past each other a bit — for the specific technical portion of the interview where you are assessing coding ability (the thing you can leetcode to practice for), the resume doesn’t really add much value and can serve to bias the interviewer.
There is usually an experience and goals interview where a manager interviews the candidate to see if they would make a good fit for the team. That definitely involves a resume and a discussion of past work experience with the objective of confirm if they’re a cog or a tech lead :)
It can be important to split them up especially if you’re relying on more junior technical interviewers to validate coding skill. They won’t always know what they’re looking for yet along this axis.
> It’s become “grind leetcode” which is clearly a failure mode.
I have no insider knowledge, but I always wondered if those kinds of cultures had a hazing / blind-leading-the-blind aspect to them. I.e. the people who got hired were the ones who jumped through some arbitrary hoops, so they doubled down on those hoops being the right hoops to choose the best candidates.
You see the same effect in the cottage industry built around "cracking the code" and getting hired in these sort of companies. They produce marketing/video content that endlessly repeats "grind leetcode and 3 other simple tips" for getting hired, and then they insist that this is the exact and only reason they got hired (rather than plain luck or some reference), so everyone outside looking in emulates and internalizes this idea as the exact and only way hiring could possibly be effective and "fair". Eventually those practices just become the norm and people are blind to alternatives.
What was once considered a test of "intelligence" and skill in coding (whether a true assumption or not) has just become a test of "how much is a person willing to struggle to earn entry into our hallowed halls". Actual potential ability in the role is secondary to getting the role.
This is a great point. As companies get bigger you can find HR actually codifies the behavior as part of their efforts to reduce unconscious bias in the hiring process. So you end up with developers, who know this is a flawed system, perpetuating it because the rules require it.
Hazing is exactly what it is. It's a club and if you want to join the club you're going to have to go through the rituals. I hate participating in the charade but I do it with as much humanity and compassion as I can muster.
I work at a FAANG (and obviously, I'm not a company spokesperson, just sharing my own experience). Those who are passionate about interviewing internally all seem to agree on not asking leetcode questions. I know leetcode questions get asked anyway, but there's pretty clear internal guidance and training for interviewers saying not to use them.
At least part of the problem is that leetcode questions are easy to ask, and most interviewers don't want to go through the hassle of coming up a question that scales well to the candidate's experience and knowledge.
Leetcode has the side-effect of filtering on “can code yes/no” which is a completely fair filter… I just wish it wasn’t difficult DSA problems we used.
The same screen can be done with much simpler problems. This light coding interview should also be more or less pass/fail. Do it before everything else to short circuit those who have no idea how to write a for loop.
Save the interesting insights for real problem solving, code reviews, systems design… DSA under pressure is not something that actually happens on the job IME.
Still, I’m conflicted, because a solid DSA understanding is incredibly helpful at times. At the very least, a solid understanding of the commonly used data structures, how to transform data structures, a good understanding of memory allocation with respect to code and runtime, reference vs copy… All things that if not understood, can cause serious problems.
> Still, I’m conflicted, because a solid DSA understanding is incredibly helpful at times. At the very least, a solid understanding of the commonly used data structures, how to transform data structures, a good understanding of memory allocation with respect to code and runtime, reference vs copy… All things that if not understood, can cause serious problems.
The same screen can be done with much simpler problems. Or direct questions.
The trouble is it’s a hard problem, and it usually gives -some signal, so it’s sort of better than nothing? I guess? In cases where contract-to-hire make sense for both the company and candidate I generally regard that as ideal, but that’s not every situation.
Someone will solve this, and that person will be very well-loved.