Hacker News new | past | comments | ask | show | jobs | submit login
Senior devs. Is anyone else insulted by coding exams?
147 points by dtagames on March 9, 2022 | hide | past | favorite | 394 comments
After an invitation to apply for a job with a YC company yesterday (an email from the CEO offering to "have a conversation") I got a generic invite to take an online coding test in Javascript.

I've written JS and shipped numerous apps in it over the last 20 years and let me tell ya, if anyone worked for me and wrote code like I saw on those tests, I'd fire them.

My favorite was a sort question that had an obvious bug. Both sides of the sort had the same test (so no sorting would happen). But the question wasn't about that... it was about which input would tell you the code was wrong. This is most ridiculous way to diagnose or fix a bug that I've heard of.

Nobody who really works on software works on these kind of bugs. And this diagnostic technique of arbitrarily working backwards to force a multiple choice solution is probably the worst method of programming that I've ever come across. No wonder so many companies are hiring shitty programmers and writing garbage code.

Without an actual problem and an understanding of the problem domain, no programmer is going to write anything useful for you. Frankly, I'm frustrated by this method of interviewing and selecting candidates. It makes me not want to work at places that do this.

Anyone else have a similar experience? Also, hiring managers... Do you find these kind of tests useful? Why? How?




I've interviewed hundreds of programmers, and really, you cannot trust pretty much anyone.

There's so many people that think they can code, or just say they do that really don't, that some sort of basic coding test is necessary if the job requires it.

Our hiring rate for some roles is 200:1, but we pay handsomely for those.

Think it the other way around, interviews are a two way process, you're also interviewing the company, and the way they conduct the interviews, in many ways also tells you what they value. What they're like.

Take it as an opportunity to get to know each other. Bring this up in a constructive manner. It may make all the difference and provide a stronger signal to both of you, you learn how do they react to feedback and at the same time you show that you're capable of critical thinking and care enough about your craft to call bullshit when you see it.


If those tests actually filtered out incompetent programmers, I'd say they were great, but my experience is that they all test very shallow knowledge that doesn't relate to ability to produce software that performs under load.

If you could come up with such a test, it ought to just be used as a standard for everybody to take.

And yes, as unpopular an opinion as it is around here, I'd love to see a "bar exam"/licensing board for software developers.


I've had moderate success discussing Rust (a language we hire for) at moderately low levels.

Ie, i've had the thought that it's difficult to bullshit topics with any depth. Even basic questions, why isn't an Rc safe to send to a thread? How does an Rc work? What is the difference between an Rc and an Arc? etc

I'm happy to accept "i don't know", those are valuable too. But the whole conversation process led me to believe a lot about the candidate. We've hired 10+ juniors and seniors with this process and i've not felt cheated yet.

I still feel like i need to find a way to open up people who "freeze". I do that myself, so i can relate, and my strategy of interviewing is pretty 1:1. Still though, i've had more faith in it than live/take-home tests.


> I'd love to see a "bar exam"/licensing board for software developers.

Can't resolve IRQ conflicts? Don't know the big-O complexity of different Delaunay-triangulation algorithms? Out of date on which features Microsoft added to LINQ in different versions of C#? Don't know where the "launch debugger" button is in Android Studio? Sorry, we aren't allowed to hire you to work on Instagram's iPhone app.

Better hope the police don't find out your Android has an unlocked bootloader; I hear prosecutors are really cracking down on unlicensed software development ever since the assassination. They say unregulated software development leaked his location to the Zetas, I don't know if I believe that.


Witness the world of unlicensed renegade civil engineers roving across the American landscape, building unauthorized bridges and infrastructure.

Or perhaps Robert De Niro's character in Brazil.


And I think this is where the crux of it is; not all software is equal.

Building bridges is very different from putting a shed in someones backyard.

Software is a spectrum and if your company builds sheds in peoples backyards then you don't have to hire in the same way you'd hire a civil engineer.

Lots of companies copy the Google style interview without actually being anything like Google or any understanding why Google hires that way.


Man, can you imagine how great our physical infrastructure would be if the Linus Torvaldses and Fabrice Bellards of the civil-engineering world could actually do this?

Now imagine it was illegal for Linus Torvalds and Fabrice Bellard to do what they've done.


https://www.bbc.com/news/world-asia-china-27274086


It's difficult to interpret your comment. It's a link to a news story about how a partly built "illegal" stone bridge collapsed, killing 11 people, presumably because it was poorly designed. In context, you seem to be suggesting that Linux, QEMU, and ffmpeg are analogous to that bridge because they are of extremely poor quality, while there's some kind of official government software licensing body that protects people from such terrible software. Is that what you meant?

No such body exists, though many attempts have been made to establish one, with dismayingly destructive results, as I outline in https://news.ycombinator.com/item?id=30621151.

I think that if we want to enable the Fabrice Bellard of civil engineering to build bridges, we probably need to keep all humans at a safe distance from the bridge until it's fully complete, then stress-test it to validate the FEM model before opening it to traffic. Also, probably not build it out of brittle materials like stone, because the stress test wouldn't give you enough information to validate the model's failure predictions.

We should be thinking about how to make civil engineering more like software engineering, with rigorous version control, automated testing, automated builds, rapid improvement in the state of the art, and safety features that minimize risk to humans in case of failure. Not vice versa.

(Though it would be nice if more programmers were in the habit of planning ahead and thinking about failure modes.)


It's an unserious comment responding to an unserious discussion. You've provided a bevy of historical knowledge, which while interesting, in no way resembles what software engineering licensing would be like in the present day. For one thing, corporate power is so entrenched and regulatory bodies so enervated in this space (as everywhere), there is absolutely no possible way the licensure dystopia you've so vividly conjured could exist. And nobody on this forum calling for licensing is asking for some onerous top-down governmental bureaucratic licensing regime, so you've basically created a neat '90s cyberpunk retrofuturistic setting that has little to no relevance to the discussion at hand.

If anything, people are calling for industry standards, something akin to craftsmen guilds of olde, overseen by an existing professional association of engineers such as the ACM or the IEEE [0]. Alternatively, perhaps it could arise as the end result of Triplebyte or some other private solution trying to be the Collegeboard of this industry. Speaking of which, academic exams such as the SAT, the ACT, or AP tests have no legal binding, but most higher academic institutions (until recently) choose to use them as standards. So it's more of a widely-subscribed informal standard.

In practice, such a license wouldn't hold too much water anyway, certainly FAANG would ignore it for the most part in favor of their higher standards, but perhaps it could at least be a Leetcode replacement for smaller firms [1]. That would save both applicants time in having to run the LC gauntlet over and over again and every single company they interview with, and help those smaller companies avoid their hiring woes of cargo-culting FAANG interview practices and accidentally filtering out every applicant along the way. None of this would forbid the practice of programming by non-accredited practitioners, as the licensing regime bogeyman you fear would have absolutely no teeth for enforcement.

Finally, if formalizing the engineering nature of software engineers means that American engineers can be like their Canadian counterparts and wear iron rings [2] to mark their achievement and profession, then that is infinitely more cool than the status quo and more compelling than any number of fantasy dystopias, or evocations of the state of the art of '70s computing, or Operation Sundevil, or the goings-ons in... Iran.

[0] https://news.ycombinator.com/item?id=30616114

[1] https://news.ycombinator.com/item?id=30493041

[2] https://en.wikipedia.org/wiki/Iron_Ring

> We should be thinking about how to make civil engineering more like software engineering, with rigorous version control, automated testing, automated builds, rapid improvement in the state of the art, and safety features that minimize risk to humans in case of failure. Not vice versa.

Have you seen the state of software engineering? I'm not sure if this field is in any state to be acting smarter-than-thou towards other engineering disciplines.


> If anything, people are calling for industry standards

Industry certifications do exist. They just haven't been very successful so far. Perhaps we're going to see more focus on them if the pace of development and disruption slows down, but as it is I don't think there's much interest in them.


Not only could it save us time on job interviews, it could save us time on tech support calls: https://xkcd.com/806/

I think you're being unrealistically conservative about how much things are going to change. You might be right that things will get better instead of worse, but overall I think you're seriously underestimating just how much and how fast the world changes over time. Don't forget that it's only been 21 years since "Know Your Customer" was an unthinkable Big-Brother sort of idea, only a year since "vaccine passports" were widely thought to be a paranoid fantasy that could only happen in Israel, and only two weeks since Russia invading Ukraine was widely dismissed as implausible.

More broadly, I think that if you want to know what's going to happen 50 years in the future, you need to look at things that have happened at least 50 years into the past. 02072 will probably be as strange to us, if we live to see it, as our life today would have seemed in 01972.

I'm not sure why you're dismissing Iran. 85 million people live there and their welfare is important.

As for "smarter-than-thou", I think it's rather that good programmers are acutely aware of the severe limits on human mental capacity and therefore look for solutions that don't require them to be infallible or even smart. Also, we have computers, so we can build our practices around them!

A couple of years ago, Hillel Wayne interviewed a number of "crossover" engineers who had worked both as software engineers and in other engineering fields, and my recommendations above are strongly influenced by what they said about what the different fields can learn from one another. If you're interested in these issues, you might be interested in reading his writeup: https://www.hillelwayne.com/post/what-we-can-learn/


Actually, if you had wanted to debunk my proposal, this classic would've be the one to do it: https://xkcd.com/927/

> I think you're being unrealistically conservative about how much things are going to change.

They will change, but the direction you are invoking is fundamentally in the opposite course of history wrt this industry. When has there been greater government control over the software industry, specifically when it comes to what constitutes a software engineer?

> You might be right that things will get better instead of worse

I said corporate control is rampant and regulators are as weak as ever. I supposed from a certain political point of view things are getting better.

> only two weeks since Russia invading Ukraine was widely dismissed as implausible

The Orange Revolution happened in 2005. I'm sure technothriller novelists and wargamers have at least been considering the current scenario since then.

> I'm not sure why you're dismissing Iran. 85 million people live there and their welfare is important.

Because this entire discussion has been mostly about Silicon Valley-adjacent engineering. Perhaps Leetcode interviews are a pain point for Iranian programmers. But if you're bringing up government tyranny in civil rights in one area to dismiss a licensure scheme - or more accurately, certification [0] is just plain ol' motte-and-bailey.

[0] https://news.ycombinator.com/item?id=30493814

> Also, we have computers, so we can build our practices around them!

Computers are only as good as those who write the code, unless we are to start building our practices around the results of neural networks, which is a most intriguing proposal.

That Hillel Wayne article is a good read, but again, I am calling for humility. Because much of the engineering in this industry is far from the lofty bridges and oil rigs the word implies: https://www.karllhughes.com/posts/plumbing

And to return to Wayne, consider part one of his series: https://www.hillelwayne.com/post/are-we-really-engineers/


Why would I want to debunk your proposal? I don't think it's bunk, I just think it would have some undesirable consequences. That's no bar to pointing out the desirable ones!

This is the first time Silicon Valley has been brought up in this discussion, but keep in mind that practices in the US are influential around the world, and the harms from strengthening inequality and substituting credentialism for meritocracy fall predominantly on people outside the US. The discussion has mentioned YC (Boston originally), Rust (Mozilla, San Francisco, previously SV), Microsoft (Washington), Android (SV), iPhone (SV), Instagram (SF), the Zetas (Mexico), Brazil (the movie), the American landscape (much of which is in Brazil, the country), Google (SV), FAANG (SV/Washington/SV/SV/SV, but all with offices worldwide), Linus Torvalds as a student (Finland), Fabrice Bellard (France), Guangdong (China), the ACM (US-wide), the IEEE (also), Triplebyte (SV but kind of worldwide), the SAT/ACT/AP tests (US-wide), Ukraine, Russia, and Canada.

I agree that the danger of war in Ukraine was foreseeable; I wrote about it in 02016 https://dercuano.github.io/notes/wwiii-genesis.html https://dercuano.github.io/notes/state-of-the-world.html and 02017 https://dercuano.github.io/notes/2017-sap-allocation.html and George Kennan wrote about it last millennium. But it's also true that it was widely dismissed two weeks ago.

I'm glad we agree on calling for humility! And, yes, Wayne's whole series is worth reading.


On the same note of this prior conversation, what timing for this to be #1 on the front page:

https://news.ycombinator.com/item?id=30623617


Heh!


Yep, that's exactly the sort of ridiculous hyperbole that gets thrown around every time somebody suggests any sort of standardization around professional software development!


There's no reason to think it's hyperbolic or ridiculous.

We've seen lots of sorts of standardization around professional software development, with uniformly dismal results, and lots of criminalization of experimentation by astoundingly incompetent authorities.

— ⁂ —

The first official standard for professional software development was COBOL, shortly followed by JOVIAL.

In the 01970s the US military standardized on Ada for, among other things, their hard-real-time systems; it took about a decade of violating the standard to convert Ada into a system that was somewhat suitable for hard-real-time systems, after which an Ada-induced problem destroyed the first Ariane 5.

The ACM withdrew from the SWEBOK process after it became apparent that knowledge of actual programming was going to be completely excluded; instead SWEBOK certifies that you endorse the obsolete, counterproductive management processes to which "agile development" was conceived as an antidote. (If you're familiar with the BOK for some other branch of engineering, you may be aware that, for example, the structural engineering BOK includes not only project engineering practices but also what makes structures stand up or fall down, and the civil engineering BOK includes things like engineering mechanics, strength of materials, and chemistry. Presumably this is because political decisionmakers have been able to observe the difference between successful and unsuccessful skyscrapers and dams and infer which engineers were responsible.)

The A+ certification for IT work is impossible to get if you only know Linux and not Microsoft Windows, and last I saw, it did still require that you knew how to debug IRQ conflicts—long after ISA buses were limited to PC104 systems.

PCI-DSS audits routinely, though not universally, mandate counterproductive security practices like installing virus scanners on Linux servers.

If we'd effectively standardized professional software development in 01960 we'd still be programming in COBOL today.

Moreover, the Board of Licensure could revoke our licenses when we criticized widespread bad software development practices, as has happened to Charles Marohn of Strong Towns: https://www.strongtowns.org/journal/2021/5/23/lawsuit

— ⁂ —

You may not have commemorated it, but on March 1 we passed the 32nd anniversary of the Secret Service raiding Steve Jackson Games for publishing a "manual for computer crime", namely, the Cyberpunk module/game for the GURPS role-playing game/system.

Outside the US the situation is often even worse; Saeed Malekpour was sentenced to death in 02010 because he wrote open-source image-uploading software and a porn site used it without his knowledge. (His sentence was suspended; he escaped in 02019.)

Do you really think it would be a good idea for the governments that chose those prosecutors to decide who gets to program and how?

— ⁂ —

The bigger issue is that computers are now our exocortices; we've offloaded significant amounts of our thinking and communication onto them, as we did previously with inventions like books and clocks. Computers vastly extend our mental abilities. Without control over our computers—without the legal and practical ability to program them and to investigate what they are programmed to do—we in effect lose our freedom of speech, association, and thought. That power is far too dangerous to allow any guild or faction to monopolize it.


I mean, you don't need a really hard test to weed out most hacks. In any case, if the test is hard, companies that know their shit in hiring, care more about how you approach it than what the end result is.

In the end, the interview tries to figure out how it's like to work with you. Do we get along? Can we communicate effectively? Will you be happy and thrive here?


You sound like a great hiring manager/interviewer. However, I have read stories of people who got rejected only because they did not finish the coding challenge in time. The engineer reported that the candidate communicated well, but did not finish. Hiring manager rejects the candidate.


Yeah, there are bad employers and bad interviewers. Sometimes the importance of it is underplayed. I always drill others that interview duty is the most important thing you can do at a company (even though it sometimes feels like a chore). The people you help choose, influences culture and long term viability of the company. Take it seriously and be fair with candidates.

Having said that, figuring out a fair and effective interview process is HARD. It requires constant tweaking and monitoring. Also, you have to start from a decent set of values at the company.

For example, we do shadow interviews. When we certify interviewers, we have them come to watch and listen to an interview (we explain to the candidate beforehand what's going on), and then after the interview ends, we meet with the shadow interviewer and discuss what they picked up, and explain why I went such and such way, what I was looking for, why such and such mistake is not a big deal, and so on.

After a couple of these, we have them interview, but with a certified interviewer shadowing them. Just observing (we have them practice on us their interview exercises if any before the interview).

And the we give feedback, you rushed a bit here, give them more time to be at ease with you, did you notice such and such, whatever... If everything goes well, we certify them, and they can fly solo.

We try to have a varied pool of interviewers to avoid biases. The downside is that you need to do more interviews as a candidate, to get a fair assessment (i.e. no single person can derail you).

In the end, we've all been in a crappy interviews and try our best to not inflict that on other people.


> ... they all test very shallow knowledge that doesn't relate to ability to produce software that performs under load.

That's a later filter. The first filter is, can they code at all. Like, FizzBuzz level of "at all".

I mean, if you need them to produce code that performs under load, you'd better interview for that too. But people tend to first worry about whether they can clear the lowest bar before they proceed further.


It is difficult to comprehend how low-skilled the shallow-end of the applicant pool is. I think it’s gotten worse, not better, since the original FizzBuzz essay was written.


They actually do.

The test isn't (or shouldn't be) that you understand string manipulation (or whatever) but rather that you can work through a new problem. There are many programmers who claim many years of experience but have poor problem solving skills. They can implement a CRUD form and basic tasks, but fall flat when faced with a problem they've never encountered before.


The thing about the license is that it avoids the redundancy of having to pass the same orthogonal algorithm questions at every single company an applicant chooses to apply at. That alone is a great time-saver, and allows applicants the freedom to focus on actual work.


Not really. You'll still have interviews, they'll just be more personality and resume-driven.

Personally, I'll take writing FizzBuzz over and over again vs. "where do you see yourself in 5 years?" ("I don't know but probably not working here") and "why do you want to work for our company?" ("to earn money").


Most interview loops include Leetcode for the technical screen, then at least one domain-specific coding section (like a more tense pair programming section) during the on-site, and maybe another algorithms section on the on-site (several if you’re Google). The license would at least eliminate the algorithms section to focus on the domain-specific, which is at least more reflective of real-world work.


Also, its a lot easier to study leetcode problems than it is to study for domain-specific ones (since there are many different domains). I would expect having more domain-specific problems would limit you to a narrower range of jobs.


It all comes down to philosophy- one could argue that it's better to have interviews reflect the actual material of the work. And that such domain-specific questions would require less prep, as the candidate's experience should already be strengthening them to answer such questions. Not to mention additional career development in those skills- asking and answering Stack Overflow questions, technical blogging, side projects using the same stack, etc.


That's much worse than leetcode problems because it requires tons of work outside work. Most people, myself included, have a life and interests outside of the very narrow focus of their career, and don't want to spend 16 hours a day focused on that one aspect of life. Studying a few leetcode problems is no big deal in comparison.


Depends on the companies you're applying to, it's rarely just "a few." One of the difficulties is retention. A LeetCode a day only goes so far until you have to repeat older problems. Better to get it all done in one test.


You're assuming they will actually eliminate it though rather than just pick something else to test.


Sure, but at least that might be something more relevant to day to day work, and require less grinding than Leetcode.


> where do you see yourself in 5 years

Mostly in mirrors. Unless I turn into a vampire.


I think you are looking at this the wrong way. the test isn't too find who can write software that performs under load. It is too find out who can write software, period. Essentially the fizzbuzz test. There are a ton of people who say they can write code, but then can't write a simple for loop.


Oops, I posted my rant[1] about our need for a bar exam before realizing you got there first. Fully agree, this profession really needs this.

1: https://news.ycombinator.com/item?id=30616361


> you cannot trust pretty much anyone.

This. I hire financial analysts regularly and it's similar in that it requires a high degree of proficiency in Microsoft Excel. People consistently over many years of hiring absolutely do lie about their level of proficiency. Sometimes it's over confidence, ignorance, but often just a straight lie. So, we do Excel modeling similar to this. It's truly the only way to get a good gauge of where the individual is in their proficiency and how they approach modeling problems. I don't see a way of eliminating them, but to OP's point, I do try to make the tests generic (no industry/business knowledge required), open ended (allowing them to solve the problem however they want) and generally "good". We always tell folks taking the test, "it's not meant to be trick questions, if the instruction is confusing ask us for clarification" and afterwards we discuss "how do you think you performed?", "what did you think?" those kinds of things to try to iterate on any of the poorly constructed aspects we may be blind to.


At least with Financial Analysts they actually use Excel daily (or often).

For me I have over 20 years of experience as lead architect, designer, engineer, of both apps and databases at a wide range of companies and industries. But it feels like that experience is out of touch with how most companies hire in the current market.

Whenever I get invited for FAANG interviews the recruiters tell me it's best to cram for months on stuff I would never/rarely use in real life and learned in theory over 20 years ago. I did one interview after prepping for an hour or two per day for two months and passed most interviewers but missed on one so they told me to try again in 6 months. Non-FAANG tech companies out of Silicon Valley seems just as bad from what recruiters have been sending me.

Although some of them FAANG companies pay so well (particularly META) it can make it worth it. That said after wasting so much time on just for the option to interview every 6 months again makes me wish I could do some other industry that pays well or better.

The current interview process makes me lose my interest in the profession but not actual development of apps and sites.


Honestly, what you describe is a major reason I never pursued software engineering as a profession. I love it as a hobby and entrepreneurial tool.

Also, I kind of just prefer building my own things instead of working on some over complicated stack I didn’t design.


Is there a video demonstrating the kind of modeling you're talking about? The only thing I've seen that sounds similar was a Martin Shkreli video which is of course removed from YouTube now, and I don't know whether Shkreli was a beginner or an expert, just that he navigated Excel 20 times faster than I do.


There's some videos of high level skills if you search "excel modeling championship". Shkreli was a hedge fund/wall street guy if I remember correctly and probably has more of a banking background. That area of finance tends to be hyper competitive on Excel skills. I've heard stories of them banning mice from the office to force people to use keyboard shortcuts and what not. It's locker room braggadocious badge of honor in that world. But it's the type of application navigation that it sounds like you're describing.

I on the other hand am in realm of corporate finance. Think CFOs, Controllers, and their teams. It's a bit more relaxed here but you still need enough skill to get the job done effectively. We don't really care how you get the job done (use a mouse, keyboard, google, it's all game) but you need a nice output and your approach in an interview context should convince me that you'd be capable of solving similar problems in the future. For that reason, I typically provide crappy raw data (oddly and inconsistently formatted) and ask them to build or analyze it. Given some fictional general ledger data; can they build a 3 statement model? Can they summarize some payroll data in a logical way? Build a budget scenario based on the following actual financial data... also, here are 5 basic features the model should incorporate. I don't typically watch them so I don't know how they navigate the application (I don't find it important) but we give a time limit and stop them at time and review work attempted. The highest skilled folks tend to finish within time.

It's counter-intuitive but the worst thing for an analyst is to work at a company that has a great ERP/BI tools with reliable data, etc. It's only about 10% of companies that have their shit together in that regard and those analyst just run reports from XYZ GUI and never have to work with data. So they have low level of Excel skills but think they are great analysts. I've never actually experienced that world, but I know their analysts wouldn't cut it in the real world of crappy data and ad-hoc work. I've seen them crash and burn too many times and it's painful for the entire team.

Back to your original point, as you might expect, most folks that are great at Excel just use it a lot to build complex things. They may have learned the keyboard shortcuts along the way; or not. It just takes practice and muscle memory will kick in but also requires regular practice to keep sharp at it. There's also some pretty decent tooling to help you step your game up if you wish. I personally like https://macabacus.com/


> There's some videos of high level skills if you search "excel modeling championship"

From a few months ago...

Excel World Championship Finals - https://news.ycombinator.com/item?id=29521324


This is great, thanks!


I've recently had multiple candidates recently with inflated resumes. Not mere embellishments, but claiming advanced degrees in computer science when they can't solve a simple algorithmic problem (something that can be coded in <10 mins).

I dislike most take home problems, especially longer ones. But there's few exceptions where I could move forward with a candidate who refuses a technical interview. In almost all cases it's egotistical and a glaring red flag.


> I've recently had multiple candidates recently with inflated resumes. Not mere embellishments, but claiming advanced degrees in computer science when they can't solve a simple algorithmic problem (something that can be coded in <10 mins).

I've a BSc in one of the better universities for CS in the UK. I can barely implement any algorithm that I learned during my university years. The amount of times I've been told implement algorithm x, y, and z in the past ~10 odd years in my professional career has been zero. Conversely, the amount of times I've been asked to implement them during interviews has been way higher.

Unless you're hiring someone to implement such an algorithm, you shouldn't be using them as a test, because most, if not all, the algorithms we got taught during my CS degree, have one or more implementations to the point where I can just use those implementations. Heck, I wouldn't even want to waste time re-implementing it, and probably doing a bad job of it, when I can just use someone else implementation, who most likely will do it better than anything most people can.


No OP, but when I ask people to solve an algorithmic problem, I don’t mean “regurgitate this algorithm from memory”, I mean “I give the candidate a plain english description of an algorithm and ask them to turn it into code” (eg “read this .txt file, for each word in the file, reverse the word, and print it to stdout”). This does a pretty good job of filtering out people who can’t write a `for` loop, which is a lot of people, even those with “senior” in their past job titles...


This is exactly what I'm referring to. I'm not testing for rote learning. I'm testing problem solving ability.


The vast majority of places are asking algorithmic questions not for you to reimplement known algorithms, asking candidates to regurgitate existing algorithms is widely known as bad practice.


> The vast majority of places are asking algorithmic questions not for you to reimplement known algorithms

So your claim is every company is coming up with new algorithms that they use for their tests??


An algorithmic question is a coding problem that focuses on problem solving, and may include questions/constraints related to time & space optimization.

The test should never rely on rote learning (e.g. implement Dijkstra's algorithm from memory) or require a "magic" solution (e.g. Project Euler)


> Not mere embellishments, but claiming advanced degrees in computer science when they can't solve a simple algorithmic problem (something that can be coded in <10 mins).

I thought the second part of the sentence would be about how they don't actually have those degrees. If you say that you have the degree you have, that's not embellishing at all, and it doesn't mean that you'll be able to solve a coding problem. They're two separate things and you've kind of mixed them up.

There's two questions

- is the candidate outright providing false information (where they've worked or studied, how much experience they have and with what tools etc)

- does the true information give you the wrong impression about the candidate i.e. you think someone with this degree or with that many years of experience should be capable of more than they are


Ok I agree with your point that "embellishment" may not be the correct term and I really meant the latter.

I 100% expect that someone with an advanced computer science degree can demonstrate they have problem solving ability. I don't really think that is an unreasonable assumption. I mean we're talking undergraduate level skills that are the bare minimum for an entry level position at my org.

If it really is the case that an advanced computer science degree does not convey computer science knowledge then it reinforces the need for screening questions.


> Ok I agree with your point that "embellishment" may not be the correct term and I really meant the latter.

There's no 'may' involved. It is not the correct term.

Here in the UK, embellishing your CV can be classed as 'fraud by false representation', which carries a maximum 10-year jail sentence.


The accusation of being “egotistical” is total projection. If you want people to do monkey tricks for, the least you could do is allow a bit of dignity by letting them speak for themselves.


If someone chooses to refuse a technical interview they did speak for themselves. Even more so if they, as OP did, explicitly said that it is "insulting" or otherwise beneath them.


Stats from the experts show that even great engineers fail technical interviews [1]. Anecdotally: I've bombed interviews myself! Sometimes due to anxiety (a real problem for some of us), sometimes because the problem was difficult. So false negatives are very common. Especially if you are interviewing 200 people - you are wasting SO MUCH TIME. I simply can't believe that you're actually getting that 1 in 200 dev. Why?

Because devs also pass interviews they have no business passing. You get a technical question that you already know the answer to and so you get through it easily (this has happened to me as well). I'd consider this a false positive. And today, I suspect this is just as likely as a false negative - we have many books on coding interview prep, probably hundreds of sites with coding problems to practice. Are you getting a good dev or are you getting someone that practiced 20 questions on the same subject last week so they happen to know exactly how to solve your interview problem?

And the worst types of false positives are not ineptitude, in my opinion. I can make a mediocre dev (or a dev who is bad at algorithms) into an excellent dev. But there are a lot of devs who are very smart, but very lazy or toxic. And those people will sometimes impress you the most in interviews... because they have to do it a lot because they don't last.

If anything, if the industry insists on technical coding challenges, we should stop the random bullshit and commit to a standardized exam and license. That's how other professions do it (and it works quite well with rare exceptions), and it would save software engineers a lot of time and headache, and make the hiring process smoother for everyone.

Experience should count the most. How recently have they been coding. Look for signals like: they got promoted. Listen to them talk about things they worked on and find out how much detail they can give on those things.

1. https://medium.com/free-code-camp/you-will-randomly-bomb-tec...


> Listen to them talk about things they worked on and find out how much detail they can give on those things.

This always stresses me out, not just in interviews (which are actually a bit easier since you try to prepare beforehand there, although I guess the anxiety and stress can cancel this out anyway) but also in everyday chit-chat with other devs. I strongly suspect that I may have ADHD and I'm just unable to randomly talk about things I've been doing even as recently as last week without preparation. I can recall a lot if I actually sit down and calmly trace back some stuff I've been doing years ago, but if I got surprised with a question like "you said in your CV that you used X at job A, what was the thing you liked the most about it?" it could take me the whole interview to just get into the mental state necessary to even recall my experiences with X, let alone put them into proper words. I'll eventually find a perfect answer, but chances are high that it will happen at evening under shower accompanied with a huge regret titled "I should have said that back then" :P


Um, not to sound like an ass but how is being asked to dig into the things on your resume a surprise?


Every single thing on it? I've worked with dozens of technologies across multiple fields in my career and I'm still in my (late) twenties, so there's probably more to come. I don't think I could be well prepared for every single question about my past that could come up during an interview. I once got surprised with a question about "the story that led to" discovering a CVE that's attributed to me, for example. In an asynchronous chat that would be a very fun question to answer, but not in a real-time conversation. Playing a detective trying to reconstruct your own memories clue by clue isn't compatible with keeping the interview going.

Nevertheless, I can be prepared on what I was working on during my time at company X, but a question like "what I liked/disliked the most" could still put me off track if I haven't thought about it before. I just don't know without a longer analysis.


I've actually prepared a document with common interview questions / answers / examples of work to talk about, which I typically review before an interview and keep open in my browser while I'm doing video interviews. I think it helps, you might find it to be a useful exercise.


>Anecdotally: I've bombed interviews myself! Sometimes due to anxiety

I used to get pulled in on anxious candidates to help make a final decision. Interviews are rarely an environment were a person does its best work. Experienced interviewers know this.

Typically, I didn't make them code. Coding and anxiety don't mix well.

I usually approached the interview with roughly the following structure:

- Reduce anxiety

- Work on a problem (figure out how it is to work together)

- Reverse interview

To reduce anxiety, what has worked for me is basically making them feel in control. I first make them talk about themselves (i.e. something they know), and then I try to find a passion, something they feel strongly about, related to a project they did (hobby, uni, whatever).

Have them walk me through it (still something they know and control the narrative on, and also something they care about).

I hear actively and engage with them. I'm non judgmental and positive about whatever they tell me.

At this point most people feel at ease.

Then I present a simple, open ended design problem, no code required, but I may probe for specific knowledge here on DB, distributed systems, concurrency, probability, security, whatever I'm looking for. Usually as a discussion of some sub-topic in the design.

I make it clear at the beginning, that I don't care much about the solution and more about the process. Picture us working together working on this problem, this is what I'm trying to figure out. How is it to work with you, and hopefully you'll figure out how is it to work with me.

Then I typically close with an interview reversal, I give them a chance to ask me anything and I answer truthfully so they can assess if they would really like to work here.

This has worked well for me, but I'm an extrovert, so the putting you at ease part is relatively easy for me (it's emotionally exhausting sometimes).

>specially if you are interviewing 200 people - you are wasting SO MUCH TIME. I simply can't believe that you're actually getting that 1 in 200 dev. Why?

Most of the 200 don't pass basic screening, 30 will do a simple code screening, maybe 10 will get to an actual interview. Then one or two of those will get hired.

>But there are a lot of devs who are very smart, but very lazy or toxic.

We care a lot about this, we've passed on really smart candidates with lots of experience because we thought:

- they wouldn't be happy working with us

- they were hard to work with

BTW we measure post-interview experience, whether you got hired or not, to improve our interview process and assess interviewer performance, we might pull somebody from the interviewer pool if they perform poorly at it.

>a standardized exam and license

I really don't think it will change anything. You have a CS degree, I will still make you code a bit.

In any case, hiring requires many signals. Coding is just one, there are many others at play that can make or break a hire.

Also, the company moment matters: are we risk averse at this point? A startup usually would rather lose a good candidate than hire a bad one, larger companies can take more risk, and give you time to prove yourself.


We've talked about this so many times on HN, and I've weighed in on this so many times. But it's an important enough topic that we should have the discussion again.

You left undefined what you mean by "some sort of basic coding test". I have been screened out of jobs based on questions like "find all matching subtrees in a binary tree", or "find all sub matrices with positive determinants in an NXM matrix" (at the whiteboard, in 45 minutes).

I don't think this is as much about critical thinking as you do. What limited feedback I got (from a recruiter, never the person who did the test) was "analysed the problem and sketched the solution well, but didn't make enough coding progress."

Again, this was 45 minutes at the whiteboard in a day of getting shuttled from room to room for 5 hours of this.

At least you didn't mention "programmers who can't write fizz buzz." I have never heard of anyone getting screened out because of fizz buzz. You may have seen it. But based on my experience, the coding problems are way, way harder than fizz buzz.

Maybe you're doing things differently. If so, that's great, but I don't believe it's he norm at all.

I'm also totally ok with you giving a difficult technical test that will weed out many competent programmers, as long as you don't pretend it isn't.

I'll finish the way I always do - the alleged "shortage" or software developers in Silicon Valley is a product of rational market behavior among educated people who are free to choose a career according to market signals and their own personal interests. Re-taking your second year undergraduate data structures and algorithms exam is about as appetizing as retaking organic chemistry for the rest of your life for a physician, or retaking your partial differential equations exam is for an actuary. If free citizens don't want an employer's job in the numbers the employer thinks they should, even though the employer has decided it pays well, that's the market's answer. I see no need for the government to ask again on the employer's behalf.


>You left undefined what you mean by "some sort of basic coding test".

Yes, they never define it, because it's really hard work to clearly define what the bar is in terms of a know coding problem (like fizz-biz).


A precise definition in the abstract would be difficult. However, examples are still very useful. Typically, there's an initial easier screening followed by a much harder coding test. Rather than discuss them in the abstract, why not provide clear examples of each?

For example, consider the difference between the three statements:

You'd be amazed with how many candidates can't actually program. Like at all.

vs.

You'd be amazed with how many supposedly 'senior' programmers can't pass a very simple programming test. And even those who do can't handle relatively elementary data structures and algorithms once they get to the second round.

vs.

You'd be amazed how many supposedly 'senior' programmers can't code up fizz buzz or sum odd numbers from 1 to 100 in a loop. And even the ones who do can't search a binary of integers to see if it contains a particular value at a whiteboard in 45 minutes.

The last one doesn't define what an easy and hard problem is in the abstract universal case, but it gives me a very good sense of who is and isn't failing these tests. Without that, the claim is relatively meaningless.

Unfortunately, I actually think that many of these claims are deliberately left ambiguous. Many people who are claiming a shortage really don't want to be clear about why.


yeah I just dont get why we have to make the coding exercises algorithmic.

I care more about like, do I have to hold your hand or not? no I dont. Okay we are cool.


I'm not a fan of leetcode questions on a whiteboard, and I personally don't ask those. However for an online automated assessment, provided the question is reasonably time-boxed and not too crazy, I think that's fine, on both side of the interviews.


The questions available at CodeWars page seem the best for me: They are no "gotcha," algorithmic questions (except the higher level ones) and they allow for a good amount of problem solving.

The ones at LeerCode are asinine... oh, you dont remember Dynamic programming? Tough luck. Even if you really wont use it at all.


It has to be simple, not too hard. That first test is basically a screening. Will I invest an engineer's hour to try to figure you out?

Then, if you get to a whiteboard challenge, in those the end result is not usually what matters, but the process of problem solving is what you typically look for.

Can we communicate effectively? Will you thrive here?


I agree there are some very good fake programmers. And I haven't interviewed as many programmers, more like tens of them but I had good luck discussing the issues I was having at work.

For example, I might tell them about a bug I am working on or even show code on my laptop. Good programmers actually wanted to solve the issue. Many solved or made useful suggestions.

Fake programmers usually had pretty hard time, even had troubles verbalizing questions to ask.

And sometime I turn tables and ask them what they are stuck at work and I try to help them. With good programmers, we had good discussion and sometimes actually solved their issues. Bad programmers never had any issues.


> 200:1

Do you mean you hire 200 out of every 201 people? Or you hire 1 out of every 201 people?


Source 200, hire 1. Many don't even get to basic screening,


> Also, hiring managers... Do you find these kind of tests useful? Why? How?

Former hiring manager.

It's unfortunate because it is a bit insulting. However, there are some people who lie. They lie about being able to code. I don't want to hire a senior dev who I then have to fire shortly thereafter.

A simple, short coding test that any senior dev should be able to do quickly in their language of choice or a longer project that they get paid for are my two favorite options to filter out folks who say they are senior devs but can't code.


How do you filter out hiring managers that can't hire?


Promoting people to management randomly might get you the best results anyway. https://www.theguardian.com/education/2010/nov/01/random-pro...


I read an analogy years ago about how stupid the logic behind most promotions in software are. Think about a sports team..

"That guy is our best player! He scores more points than anyone else in the team by far! Quick! Let's pull him out of the game and make him the coach!"

Why not keep him in the game and just look for someone who can be a good coach?


Difference is that on a sports team, you are under contract to be a player and there's not really any chance to just be promoted to being a coach.

If you have a high performing dev who wants to be a team lead and you hire someone else for that role, you risk demoralizing them and causing them to look elsewhere.

Morale is important. Promotions, real promotions with more responsibility, are important


How do you filter out pilots that cannot fly?

It doesn't always make sense to compare two occupations with each other and pretend they can use the same approach for hiring people.


>How do you filter out pilots that cannot fly?

Piloting specifically requires a qualification like being a doctor etc. These qualifications are recognized and remove the need for assessment of a skill during an interview.

However there is no standard international qualification for software development like these other professions that are restricted -- you simply can't call yourself a doctor as these professions are regulated. Anyone can call themselves a software developer. Do we really want to regulate the software dev profession?


Pilots have licenses and documented flying time. Perhaps there's something to be learned from that profession?


I can just see the posts on HN now: "Senior devs. Is anyone else insulted by the coding exams required for the dev license?". Or even better, have to log a certain amount of hours every year to keep that license.


What pilots do can be automated. I mean, it's not easy, but then again, planes nowadays can land themselves by themselves. As the automation gets better, you need pilots more and more just for emergencies, and maybe if you only use pilots for those cases, then a) your pilots don't gain the experience they need to be useful in emergencies, and b) you under-utilize them. And maybe that means you can't really automate pilots after all. But still, what pilots do is very machine-like.

What devs and mgrs do is less machine-like, and it follows that what works for "making sure you have pilots who can fly" doesn't quite translate to "making sure that you have managers who can manage".


I said there are things we can learn, not that we should adopt their pattern fully.

Actuaries, accountants, doctors, lawyers, even hair stylists all have qualifications that are required to do the job that says they can perform the basic tasks safely. What's so special about ours that we can't do the same? I hate needing to prove that I can fizzbuzz to each company individually.


We collect post interview feedback for hires and non-hires, and measure candidate NPS on the hiring process.

If some hiring manager or interviewer scores badly, well, he may end up not interviewing anymore.

We want even our non-hires to have a good experience interviewing with us.


NPS: https://en.wikipedia.org/wiki/Net_promoter_score?


Yeah. We adapt the concept of customer satisfaction to interviewee satisfaction in this case.


Thankfully, I've never had to deal with this problem.

I learned that I don't want to get better at management, so I am no longer a manager.

I assume that managers have ways to evaluate managers, but I don't know the gruesome details.


This is the question. To me it appears that hiring managers are incapable of filtering without exams -making the hiring managers skills questionable.


What's the alternative? As a Hiring Manager, do I need to spend multiple hours with 20 candidates to get a really valuable insight into who they are really like or should I just screen them with a basic exam so I only need to interview the 4 who can do it?

What insults me is when someone wants to be paid more than a doctor based on no qualifications and the fact that they tell me they are worth it. If you cannot do the exam but can discuss it with the company who accept your argument then great, if you can't then you probably don't want to work for them anyway so just chalk it up to experience and move on.


> What insults me is when someone wants to be paid more than a doctor based on no qualifications

What's stopping you from selectively removing exams from people who actually do have the qualifications?

> If you cannot do the exam but can discuss it with the company who accept your argument then great

I could flip it around and say, if you cannot hire strong engineers without exams, discuss with your company to make someone else the hiring manager.

If I was your employee and I saw positions remain open for years (burdening existing employees), I would question your ability to set good processes.


> As a Hiring Manager, do I need to spend multiple hours with 20 candidates to get a really valuable insight into who they are really like or should I just screen them with a basic exam so I only need to interview the 4 who can do it?

Why is your time more valuable than mine? Why should I invest my personal time in to your hiring process when you can't bother to spend the same amount of your on the job time, to hire someone whos going to add far more value to your company than your company is going to add to their profile?


> Why is your time more valuable than mine?

No one is forcing anyone to apply for a job at any company. As the seller of labor, you get to choose how much to invest in any one opportunity. Just like someone selling apples can choose to open a roadside stand, try to get into a grocery store, or sell direct to consumer.

The buyer, of course, gets to choose the process by which they evaluate the labor (or the apples). If you don't like how the buyer evaluates what you are selling, find another buyer.


How often does a candidate interview, maybe 1–3 companies every 2–5 years? My manager probably has a hundred candidates every year because someone in the org is always hiring.


What’s your hiring process?


Contact some references? Surely a good candidate would have some previous managers that could speak to their job performance.


I would never give out references without a contingent offer that I intend to accept. I don't want 10 different hiring managers harassing my former co-workers.

I would much rather take a short coding exam.


There is nothing wrong with a hiring manager recognising that an exam helps them make a hiring decision. What you really need to worry about are hiring managers are incapable of filtering without exams and not using exams.


If you hire someone who turns out lied on their resume, you handle it the same way every time - you retract what you have offered for the truth. Guy can't code? Fire him, hire another and take it as a risk of the process. Don't offend your entire talent pool for the actions of very, very few (I have never hired anyone or heard of this happening in 25 years) bad apples.

It might be more common now in the remote era but this has been a standard practice for years so I don't think it's a valid excuse.


> Guy can't code? Fire him, hire another and take it as a risk of the process.

There's a couple of "problems with this idea in reality" I feel I should point out:

  * The process of firing and hiring someone is often a *very* bureaucratic thing that requires many people to sign off on it.  It can often take weeks or *months*.  That's a lot of wasted time when you actually need someone to fill a role.
  * Some countries just plain don't let you do this (e.g. France).  Once you hire someone you're basically stuck with them unless you can prove they did something really, really bad.
  * It can be downright evil to just up and fire people shortly after hiring them just because they're not as efficient as you'd hoped.  You'd *think* that having a process like you suggest would only get rid of people that lied on their resume (good riddance) but in reality it'll be abused by managers to shape their reports in ways that are beneficial to them but not necessarily the business overall (zero-consequence firing).
  * There's something to be said for letting people through your hiring process that aren't qualified...  It's downright dangerous from an operational stability and information security perspective.
IMHO the real problem is HR: They suck at filtering out IT candidates. They're so bad that you'd be better off throwing darts and sending those resumes off to the hiring managers. They filter out perfectly qualified candidates that could be the best thing that ever happened to your organization and then they let through people who's only useful skill in life seems to be making a good-looking resume.


These are all excuses. You can fire someone whenever you want in the US. It can be the same day you hired them and you can even not give them severance.

I’ve seen this happen plenty enough even in SV. If someone wants you fired - they’ll do it asap.


Sure, but turnover has a strong effect on employee morale and the company’s reputation

So even if it surfaced the best candidates, hiring anyone that applied and then firing 90% of them after a week would result in a depressing work environment and an ever shrinking pool of qualified candidates (who wants to upend their life to take a job they only have a 10% chance of keeping?). Not to mention the cost of onboarding.

This is how we get to the status quo: do as much filtering as you can before you hire.


Just because you are legally allowed to do it, doesn't mean you should do it at the drop of a hat.

Slacking off during the hiring process with the intention of firing a large portion of the people you hire is a violation of the social contract and a huge asshole move.


This is true. Workers in the US have effectively zero rights to employment until they’ve been employed for at least a year, and even then you can still fire them and just make up a reason. Corporations spend a lot of lobbying dollars to make sure the US remains a corporate authoritarian state.


> IMHO the real problem is HR: They suck at filtering out IT candidates.

HR is really only qualified to filter out based on the qualifications on paper. These are sometimes misleading and occasionally outright fabrications. Those can really only be verified by someone with the appropriate technical skill to separate bullshit from an accurate representation of the abilities of the candidate.


Then why putting them in the hiring process?


They are typically the owners of the application tracking software and schedulers of interviews.

If the hiring manager wants the firehose of the "everyone who clicked quick apply" on LinkedIn or Indeed, then fair 'nuff. However, my reading of the comment that I replied to was the issue that HR wasn't filtering out unqualified applicants. Removing HR from the hiring process won't improve on that issue.


It takes time to hire someone and once you do you of course stop interviewing other candidates. Then when the person joins you typically don’t know immediately what the problem is, then once they’re gone you have to start the search all over again.

Much better to get someone qualified in the first place.

I typically use a small coding test, but not even as elaborate as described in this post — more like a 10 or 15 minute discussion/problem just to see if they really do know what std::string is or how to use an op amp or whatever. Some people can really bullshit well. Consider this the simple “do they know which end of the knife to grab when picking one up?” test.

Once that is clear, a conversation (usually just a conversation about a hypothetical problem) works well.

I don’t understand the coding tests that test your tools knowledge btw. If you know how to think and are somewhat familiar with the appropriate language then you can pick up the specific tools and frameworks. We hire for the long term so said tools and frameworks will change anyway.


>and once you do you of course stop interviewing other candidates.

Sadly, I think more and more companies are posting for jobs/positions they don't really have available/open at the moment and are just proactively trying to find potential workers (because this whole hiring process is a mess and take so long).

Hiring is becoming an always on activity.


My company has sort of been doing this, and I (engineer) have been trying to get us to stop. We do have a couple of open slots, but we're doing intake for much much more than we need.

I can offer this bit of perspective on the practice, without justifying it because I do think it's bad. We (again, engineers) spent a long time pushing for hiring to move away from the desperate search because we need someone immediately model to keep our lanes open for whenever someone awesome is looking for a job. Management finally bought into the idea...kinda. Basically, what we're doing is the mixed up interpretation of a that idea, after it passed through the bureaucracy. Instead of keeping just a line open, we're still trawling with big nets, scooping up way too many resumes and letting those people down.

The root of the problem as far as I can see is that screening is atrocious. The people at the front end of the process, the actual recruiters, have no idea what we are looking for, and the output may as well be random. I still don't know how to fix it. :/


> The root of the problem as far as I can see is that screening is atrocious. The people at the front end of the process, the actual recruiters, have no idea what we are looking for, and the output may as well be random. I still don't know how to fix it. :/

One is to have the hiring manager or someone who would be working alongside the new hire write the job description and not allow the recruiter to write it (no “17 years of experience in Airflow”).

Another is to have an engineer do a phone screen after the recruiter has screened them. Having the recruiter listen in is good as they will hopefully learn more about what is wanted.

Another is to carefully set the recruiter comp to take into account not only hires but percent that fail a phone screen.

Your recruiters aren’t your enemy but they are recruiting, not programming (you probably couldn’t do their job either) so there will always be a mismatch to some degree.


These are some really interesting ideas! Have you implemented them at one company or another? What were the results?


That seems odd to me because it’s very expensive to interview people (time spent interviewing is time not spent doing something else).

There are also people interviewing just for practice or to find a way to negotiate a raise with their management.


It is probably different where you are but what would actually happen is that it would take you at least 3 months to work out they can't do the job (they are probably not terrible just not good enough) by which time you have lost 3 months and perhaps $20K recruiters fee and now have to start the search again.

Of the people I have interviewed in the UK for "Senior .Net Developer", I would say that 75% did not meet the grade and that was with face-to-face interviews that took a lot of my time. An exam isn't perfect but if someone can't answer a reasonable spread of questions to a half-decent degree, then I don't want to hire them and it saves me hours of my time. Sorry.


"they are probably not terrible just not good enough"

Odds are, you're getting not good enough because you're using programming puzzles. Instead, why not quiz them on the work that they'll be doing, so you know that they can do it?


If you need 3 months to figure out whether the person you hired to do something can't do it then it's because you didn't really need that thing done


It’s not the actions of a few when compared to the positive hire result. Typical ratios are 1 in 100 to 400 for an SDE position. If 2% of people lie, in both cases that’s a significant number of people relative to a positive hire event.

Firing people is not as easy as you think in many places. Also, managers are human. Accusing someone of lying and going through the termination process is mentally taxing!

If a simple screen takes twenty minutes of time, it’s rational to do. If people get offended and aren’t empathetic to why you’d do this, would you really want to be on a team with them?


Yes but there are regulatory, legal and potential lawsuit issues why just firing people is not like snapping your fingers. It's an exercise with significant friction


In which country firing someone on probation would result in any legal issues? I had no problem firing people in Germany where labor laws are quite strict.


Silicon Valley, the birthplace of this type of interview, is in an at-will state, which makes the regulatory concerns quite puzzling.


And they can find an attorney to say that they were discriminated against, or that the company somehow did something improper, etc. As America is (to my knowledge) the only developed country where the losing party does not pay attorney's fees, 'winning' such a case may cost the company mid-five to six figures- or higher! Executives may be subject to invasive depositions, all company communications read by the opposing attorneys, other minor violations of our gigantic unknowable legal code found during this process, etc.


Where would a person who's looking for a job have the time and the financial resources to embark on a legal wild goose chase in search of a hypothetical payout? What kind of termination of an individual contributor for performance results in dragging in execs? This is a complete snowball effect of a scenario.


Litigators frequently work on contingency- so they work for 'free' for the litigant, in exchange for a healthy percentage (a third or higher) of any settlement. This is without including the actual government, where an employee may say that they were discriminated against somehow, and then the relevant government agency investigates everything under the sun looking for violations. For example they could allege that they were not provided their California-required lunch break, so then a nosy, hostile agency launches a huge investigation starting with lunch policies, and including anything else they find.

I agree it's a complete snowball! It's absurd :) And, it's reality- this is how litigation & workplace regulation in America works. For example depositions are so crazy, not only can you be deposed as a party in a lawsuit, you could be deposed for extra information even if you're not the one being sued (i.e. you're in a related industry as someone in a lawsuit, so the court compels you to a deposition just to talk about the industry, what's standard there, etc.)


Firing someone is much more expensive than it is to just give someone a shitty coding test and losing out on a few good devs


How do you calculate this „much more expensive“? I do not think this is correct estimate.


Just a few of the things you have to do when you have a bad hire that you fire (In the US and specifically California)

* Pay off any accrued time off

* COBRA insurance

* Unemployment

* Opening up the hiring pipeline (ads on job boards, recruiter, flights for potential candidates, training for new new employee)

* Potentially pulling other employees to fill the role while hiring again (Overtime and stress to existing employees)

Versus

Sending an email and saying no.


> Sending an email and saying no.

This is not that simple. Your hiring process is a funnel with certain conversion rates at wach step. If you overcomplicate it, your hiring timeline extends, increasing time to market for your projects and having some impact on the revenue. If you oversimplify it, risk of bad hire increases, but it may still worth it, if funnel gets much better.


Hiring someone isn't free, if only due to the opportunity cost of filling a role in your business with someone unsuitable.


I'd imagine the opportunity cost is offset by filtering out good devs with programming puzzles.


Consider that recruiters charge around 25% of 1 year of salary for helping to hire a developer, and the person-hour costs of hiring one person can easily exceed the recruiter cost for a small organization.

Firing also requires a fair amount of person-hours to do, and if you fire someone and need to replace them those costs stack.


Recruiting costs vary from one country to another. For me 25% is ridiculously high. Even considering cost of doing business, a recruiter would need to close just 5-6 open positions per year to earn as much as an engineer. If they work full time to achieve that, this is extremely inefficient for those money. Besides that, some recruiters actually offer warranty period that will cover thus specific case - you should not pay twice.


> How do you calculate this „much more expensive“?

By having actually done it, and understanding what processes are involved.

.

> I do not think this is correct estimate

It is.


> They lie about being able to code.

If someone successfully works shipping code but can't reverse a linked list, does that mean they're lying about being able to code? I've seen way to many jobs that test for expert level skills for a job that is just to add small enhancements.


> If someone successfully works shipping code but can't reverse a linked list, does that mean they're lying about being able to code?

On a whiteboard? Probably not.

In front of a computer, with a compiler and manpages handy, being able to test and debug it right away? Absolutely, yes.


> If someone successfully works shipping code but can't reverse a linked list, does that mean they're lying about being able to code?

No, I think it means they didn't prepare. But, short of examining git commits, how can you know that someone successfully "shipped code"? References, maybe (though that is harder for folks who may have fewer references from past jobs). If it is open source, sure. But how else?

> I've seen way to many jobs that test for expert level skills for a job that is just to add small enhancements.

Yes, I agree! That is a bigger issue, where the job desc/interview are disjoint from the actual work. That's a red flag. It's hard, as a hiring manager, though, to keep the job desc real. I find that I sometimes throw wish list items in ("knows <tech> at an intermediate level") because the alternative ("can learn <tech>") is so hard to test for. But as a candidate I find it extremely aggravating.

I don't know how to square that circle. If you figure it out, please let me know.

My favorite kind of interview (on both sides of the table) is where I can do the work that is typical, using the resources that would be available on the job.


> "If someone successfully works shipping code but can't reverse a linked list, does that mean they're lying about being able to code?"

Obviously yes? That's a basic assignment for a data structure class given to freshmen and sophomores. One would expect that a "senior" developer (degreed or self-taught) would be better than a college sophomore. At the FAANG level, it would be a phone screen question and a softball one at that.


Do you remember all the time in college where your professor explained the intricacies of cache hits, access latencies, and branch prediction when going over algorithms?

As it turns out, designing a fast algorithm involves a lot more than what is covered in the textbooks. This is why something like Timsort can generally outperform quicksort even though a naive algorithms course overview may well come to the opposite conclusion.

If my job is not to design these algorithms, why does it matter if I can write a naive implementation of one?


Kind of a coincidence that you should bring that up since several jobs I've been at involved stints analyzing performance and identifying bottlenecks in existing code and it is, unfortunately, rather easy to glue together even well designed libraries in ways that result in a slow, inefficient result. Developers simply cannot get away with not knowing how their tools and libraries work under the hood.


Choosing the correct algorithm is a different question from implementing the correct algorithm.


Sounds like the state of CS education should be improved to fill in those gaps of knowledge.


It matters because "algorithm" is just another word for "program". The question is whether you can think up a way to make the computer do what you want it to do at all at the level after FizzBuzz, even when it's something like reversing a linked list that can be done in two lines of code (see https://news.ycombinator.com/item?id=30616573, or seven in C), even when it doesn't need to be fast.

I mean, it's true that there's a lot of code that really is at the FizzBuzz level:

    switch (c) {
    case 'h': case 'D': menu.left(); break;
    case 'j': case 'B': menu.down(); break;
    case 'k': case 'A': menu.up(); break;
    case 'l': case 'C': menu.right(); break;
    default: /* no op */ break;
    }

    def find(self, name):
        for thing in self.contents:
            if thing.is_a(name):
                return thing

    vaders.hits(bullets, (vader, bullet) => {
        blam(vader, 2.5)
        score += vader.bounty
        vader.die()
        bullet.die()
    })

    if (vaders.where(({y}) => y > 400).count()) startGame(120)
    if (!vaders.count()) startLevel()
And I think you should write as much of your code as possible in such a painfully obvious way. Every once in a while, though, you do need to write something that can't be written quite that simply. Often, the reason why is something in your problem domain:

    async def take_token(self):
        self.update()
        while not self.tokens:
            await asyncio.sleep(self.recharge / 10)
            self.update()
        self.tokens -= 1

    // Animate explosions.
    poof.move()
    poof.each(p => {if (--p.ttl < 0) p.die()})
    // Chain-reaction explosions.
    poof.about_every(64).each(p => {
        if (poof.count() < 1024) blam(p, p.w/12)
    })
    // Limit number of explosion objects to guarantee responsivity
    if (poof.count() > 1024) poof.about_every(2).die()
You need to be able to reason through what your code will do once you've written it, and you need to be able to come up with possible ways to write it that might work. Often the code you need for your problem domain isn't already in the standard library, so you have to write it. If it's on npm, you have to be able to read it to see whether it might work.

Reversing a linked list is just about the simplest task that requires that kind of reasoning, and it has the advantage (or disadvantage) that understanding the puzzle doesn't require, or benefit from, any problem-domain knowledge. You could substitute binary search or insertion sort or something. Another example I got, which takes a few more lines of code, was "implement an integer stack ADT which can always tell you the maximum number on the stack in constant time".


I've held several senior developer positions and I'd have to study up on how to reverse a linked list. (Other than the obvious answer: "use a library/built-in function to do it", which would probably would not be a satisfactory answer.)

I think the problem is that there are over a dozen different kinds of "seniority" but we collapse them all into one term. I got so frustrated I wrote a blog post about it: https://www.mooreds.com/wordpress/archives/2812


It might depend on what the measure of "senior developer" is. There are junior-developer positions where nobody could perform adequately without being comfortable manipulate linked data structures (Linux kernel development, maybe, or working on a garbage collector or JIT compiler or the Postgres query optimizer) but senior developers on those teams probably don't know a tenth of what you know about SQL.

Also there are ancillary skills that make the problem much less difficult, so if you can't solve it, you don't have them. Joel Spolsky famously said that nobody can reverse a linked list without drawing diagrams on paper, and that's probably true if you're doing it imperatively in C. But it's in the category of "problems that become much simpler with functional styles of programming." The following took me 5 minutes, most of which was struggling with the syntax of OCaml that I'd forgotten:

    $ ocaml
            OCaml version 4.08.1

    # type 'a list = Cons of ('a * 'a list) | Nil;;
    type 'a list = Cons of ('a * 'a list) | Nil
    # Cons(3, Cons(41, Nil));;
    - : int list = Cons (3, Cons (41, Nil))
    # let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' (Cons(x, ys)) in rev lst Nil ;;
    val reverse : 'a list -> 'a list = <fun>
    # reverse(Cons(3, Cons(41, Nil)));;
    - : int list = Cons (41, Cons (3, Nil))
The full session with all my dumb errors:

    $ ocaml
            OCaml version 4.08.1

    # type 'a list = Cons of ('a * 'a list) | Nil;;
    type 'a list = Cons of ('a * 'a list) | Nil
    # Cons 3 (Cons 41 Nil) ;;
    Error: Syntax error
    # Cons(3, Cons(41, Nil));;
    - : int list = Cons (3, Cons (41, Nil))
    # let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' Cons(x, ys) ;;
    Error: Syntax error
    # let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' Cons(x, ys) in rev lst Nil ;;
    Error: The constructor Cons expects 1 argument(s),
           but is applied here to 0 argument(s)
    # let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' (Cons(x, ys)) in rev lst Nil ;;
    val reverse : 'a list -> 'a list = <fun>
    # reverse Cons(3, Cons(41, Nil));;
    Error: This function has type 'a list -> 'a list
           It is applied to too many arguments; maybe you forgot a `;'.
    # reverse(Cons(3, Cons(41, Nil)));;
    - : int list = Cons (41, Cons (3, Nil))
    #
(These error messages look appalling, but the OCaml interpreter was underlining my errors, which of course didn't paste.)

Similarly, there are problems that become much easier with other paradigms of programming, such as backtracking search.


Honestly I dont know the answer for this, its a trick you never need in the real world, but maybe it does test if you did compsci 101. I failed this as a Senior dev inverviewee, went back and learnt it so was preprared for the next time. It didn't make me a better programmer but now shows I've learned the basic tricks.


What is the trick in reversing a linked list? The solution is just some while loops...

Anyways it is a bad question to ask because it is very easy and is frequently studied so candidates just memorise the answer.


One while loop. Maybe you need to memorize again :).


There's no need to memorize anything at all; if you can code you can come up with a solution on your own within at most minutes.


I do not need to memorize it that was the point I was making, as it is so simple I can and have worked it out on the spot.


Reversing a linked list is pretty straightforward. This is going to be unpopular, but i wouldn't hire someone who can't write a program to do that.


Agreed, it is concerning 'senior' engineers are saying they can't do it.


This is why I'm convinced titles means nothing unless you're talking specific levels at a well established company. Often super duper Staff+ engineers or CTOs at no name startups often have 0 idea what they're doing


Then problem is in the questions being asking for the technical interviews.

I agree Leetcode questions are generally not a good signal because they're often removed from everyday problems one may encounter. Though they may be fun to do. If I could get a job just solving Leetcode problems, I might chose it over refactoring messy codebases or joining disparate APIs.


If someone says they are a senior dev, why can’t the HM just contact a few references to confirm? Surely that’s a better signal than a 60 minute coding test. I hate how dev hiring has thrown out references as a tool.


I'm much, much more willing to do a 15-minute programming test, or even a 60-minute one, than to give a potentially malicious company access to my network. References are for the last stages of the hiring process, once I know I can more or less trust the company.

And from my limited experience on the hiring side, while a trusted referral is worth much more than a programming test, a good report from a potentially adversarially chosen reference is worth much, much less. It's much easier for an unscrupulous person to find a friend who's willing to inflate their achievements on the phone than to provide on-the-spot solutions to programming problems they lack the skill to solve themselves.


> It's much easier for an unscrupulous person to find a friend who's willing to inflate their achievements on the phone than to provide on-the-spot solutions to programming problems they lack the skill to solve themselves.

A good hiring manager will be able to sniff that out rather quickly. Following up on references doesn't mean calling it a day after one random phone call either.


How do you sniff it out?


Obviously you give the reference a coding test


Ha! But in all fairness, multiple references, with references coming from verified previous employers. A quick phone call to the candidates prior employers to verify employment/listed reference will flush things out quickly. Also, most people are terrible liars when pressed ... even on the phone. It does require effort by the hiring manager though.


There are plenty of companies at which the senior engineering talent would very easily be outperformed by new graduates at FAANG. So calling their references to ask what their title was does nothing to verify their technical abilities.

Source: I am FAANG ATM and the new grads I work with outperform 10+ year tenured engineers at other companies I have worked at.


I assume a competent HM would dig deeper than that with a reference. It’s not perfect, neither is an online coding exam. It’s a tool that should be used in the process, that’s all I’m saying.


I was thinking about this recently and realized I haven't had to give references or have worked at a place that asked for them when hiring in at least a decade. I guess that practice is pretty much gone these days.


Same. But ‘references’ from coworkers are still a key factor in promotions at big tech companies. Go figure.


> Surely that’s a better signal than a 60 minute coding test.

It is! Where I work we have a min of three references required for non-entry level jobs and it reveals quite a bit at higher levels. It does require more effort on the part of the hiring manager though and I suspect most would rather punt it off to a test than do the due diligence when it comes down to it.


I suspect references are far worse at filtering out people than tests.


For candidates with 1-3 yrs experience, probably. OP said they have 20 years experience. For senior candidates (10+ yrs) surely references would confirm that and provide more info than a test.


OP here. More shocking to me is that I have quite a bit of published code that these folks can inspect, and running apps they can run -- in the languages and technologies they're interested in. Yet no one asks me about reviewing any of that.


Self-published repos are mutable. It tells us what you think good code looks like, but not how many iterations it took or how much help you had.


The best code is code that is iterated and looked at by more than one set of eyes. I wish all my code had been mutated to its ideal form!


Sure, but we need to evaluate you apart from everyone else who may have worked on it.


At $OLDJOB we didn't use the leetcode or whiteboard coding but we did give candidates a dead simple homework assignment (I come from infra) like "stand up a webserver with a load balancer" or "stand up a queue system with a producer and a consumer" with their preferred infra as code stack thinking we were giving candidates the chance to show off a bit but good gods the stuff we got back from people who claimed to be mid/senior were scary bad.


Homework, you say?

I've walked out on interviews that demanded this sort of homework before. I told them 'professionally' that their idea of demanding 4h of my time after 3 rounds was farcical and insulting.

Ditch the homework completely, or tell about the homework up front. Compensate - I don't work for free. And if you want a fair assessment, put me in contact with 1 of your engineers or devs and lets talk out a problem. Or review my body of work in that thing called a "resume".

If you have multiple bad hires that lied about skills they didn't have, then perhaps you ought to reconsider who is doing the hiring decisions. That's been a big one in previous orgs I've worked in. The hiring group was static, upper managerial level, and didn't understand the problem domains of the positions they were hiring for.


There's a lot of assumptions in this post.

1. We tell candidates about the homework during the phone screen which is about 20 minutes, if they want to duck after that no time wasted. No one ever has.

2. We don't compensate for it since it's the 3rd interview. The idea that I can take 4 hours of your time for a half-day whiteboard interview but not a "do a simple task that should take 30 minutes at your leisure" is weird. We've never had anyone have an issue with it because we have the one of the least intensive interview processes in the area.

3. People lie or vastly overestimate their skills on the resume, surprise. If you submit code samples or your Github is good enough then we skip the homework. Designers/artists have it right with the "portfolio" -- resumes are very nearly worthless. We don't demand code samples or FOSS projects because we don't want to assume people code after hours and obviously can't share code they wrote at work but in that case we need an alternative.

3. You are interviewed by the team you would be working with. There is no hiring group. You have contact with all the engineers. You can even talk over the assignment with the team at the end of the second interview.


> The idea that I can take 4 hours of your time for a half-day whiteboard interview but not a "do a simple task that should take 30 minutes at your leisure" is weird.

It's really not that weird at all. If you give me 4 hours of face-to-face time 1:1 with one of your engineers, I have a guarantee that you value my time as highly as you value your engineers'.

If you give me 30 mins of "homework", I have no guarantee at all, that you don't have 100 other candidates in this stage of the interview process, and one of your engineers is going to dismiss my application after 5 mins of looking at what I've produced and give me no chance of responding to any judgment they are making that I disagree with. For example I once got "I don't like the fact that you use semicolons in Python where they're optional". If someone thinks that's a valid reason not to hire me, I want the right to face my accuser.


Damn, this reads like someone who's been hurt before. I can assure you we don't give two shits about style or picky shit like that. Hell we're not even really reviewing it beyond, "okay cool they can actually write some code" and if they choose to show off then great!


Considering none of us know you, where you work, or anything; you're taking these commentaries about hiring policies very personally.

Many of us have been through terrible hiring practices. And we're sharing them here. And when you give examples or critiques, we're critiquing them right back.

I've been around the block. I've done lots of interviews in tech. So have others here (naturally!). I've noped out of many for huge semaphore-sized red flags.

But the point stands - I'm not going to do "homework" without getting paid for it. I would much rather a sit-down with a fellow engineer and talk out problems. That shows the company is also willing to put time in as well.


> resumes are very nearly worthless

Sorry for being incendiary, but this hits me as a recent trend that is nothing short of insane. I went to a top-ranked university on the highest-paid scholarship I was eligible for, got a Ph. D. there, then got the job that all my peers wanted but only a handful actually got. Then I booked 10 years of industry experience. -- You wouldn't believe the life sacrifices I've made to be able to have all of that on my resume. If someone tells me that's all "worthless" and they'd rather ask some stupid puzzle questions and see some indication of how passionate I am about working for some no-name company, or whatever the latest hype is, then that's simply where the interview ends for me.


It's "nearly" instead of "completely" because of cases like yours where there are verifiable facts that signal the candidate's capabilities.

A typical resume has none of that and is more like:

. BS from some un-ranked university

. 2 years at some company you've never heard of

.. Did things you have no way of verifying

. 2 years at another company you've never heard of

.. Did more things you have no way of verifying


You nailed it! I could copy/paste this over top every tech resume I get and it would be equally as valuable.


Your PhD is only useful if you go for a job in your specific sub-field. If you have a PhD in physics it doesn't mean you can code your way out of a hat. In fact scientist types have very poor coding practices on average.

I once had an interview with a scientist turned to industry who couldn't demonstrate how to load a text file after 30 minutes of trying. I mean, what should I have done with him? The position required lots of algorithms and data processing.


That's a prejudice that you're supporting with anecdata. Saying "PhDs make bad programmers" is not any different in that regard from saying "women make bad programmers". But at least you're agreeing with me that resumes are not worthless ;-) ...as, apparently, you find them useful for screening out people with PhDs who therefore make bad programmers.

The fact is that there is such a thing as an "impressive resume". There seem to be people out there eager to dismiss the value of those. Frequently an explanation will be that they themselves don't have one and are engaging in cognitive dissonance reduction. In fact, one can say more generally that psychological forces will act powerfully in such a way as to make hiring managers want to hire people who are just like them, even when they themselves are worse than the best candidates they could hire.

That's the reason why having an analytically well-reflected hiring process, instead of just hiring who you like, is important, and resumes have an important role to play in that.


> Here, do this work for free during your own time for a _chance_ at Yet Another Interview Round

Nah, I'm all set.


You have basically articulated why it's the last thing.


> I've walked out on interviews that demanded this sort of homework before. I told them 'professionally' that their idea of demanding 4h of my time after 3 rounds was farcical and insulting.

Haven't you read the parent post? Most people who claimed to be senior were "scary bad". In my interviewing experience it was the same - one qualified candidate in 10.


When there is open source code the candidate supposedly wrote, you could go through it with them, having them explain things, but you'd have to make sure it's not something they could have studied up if they're lying.


In Italy we have trial periods and nobody really uses them.


Trial periods are pretty common in Europe, we call it "probation" in the UK but it is hard to objectively decide if someone is up to the job. If they are terrible or brilliant, it's not a problem but in the middle, I don't want to pay some new person £20K more than another dev just because they asked for it and aren't bad enough to be fired.

Until there is something quite objective about measuring performance against some kind of qualifications, then it is in danger of being more to do with personality than genuine value.


I got the chance to peek behind the scenes once while assisting with the hiring for a startup. It was painful to see how many folks applied who had zero chance of making it. As in, they wouldn't be able to actually write a "Hello World" program, let alone FizzBuzz or the types of exams you're mentioning here. The number of those kinds of applications was up in the 80% range of all application submitted.

And HR (when you have them) are not able to filter them out properly, since all it takes is dose of buzzwords and sufficient confidence to get past them.

So, yeah, code tests are usually stupid and insulting to experienced developers. But that's because they're needed to filter out the stupid.


I've recently been doing a fair amount of reading about how craftsmanship worked in late medieval and renaissance Europe -- think crafts guilds of Florence, which were the societal context where the Medici happened and Leonardo da Vinci and many of those good things.

One of the things that jumped out at me was that gatekeeping was institutionalized at the guild-level, rather than the level of the individual company.

You only had to prove to the guild of wool once that you knew how to work with wool at the master craftsman level, and then the individual merchants and companies within the same guild would accept that more or less at face value.

If there were repeated complaints about your craftsmanship, those, too, surfaced at the guild level rather than just the individual company level, and might put an end to your guild membership.

It would be really cool if the ACM, BCS, IEEE, or any one of those professional associations, started doing something like that, i.e. you could just be an "IEEE-certified dude who friggin just knows what they're doing". Then companies could just say "people without IEEE-certification at the master craftsman level need not apply", instead of giving people exams of technical competence that are insulting to the people who will actually pass them.

On the other end of the lifecycle, an employer who finds an employee repeatedly falling short on craftsmanship could report that person to the IEEE, and they would investigate the complaint and revoke certification if appropriate.

Also: This is how it works for doctors & lawyers, and some other kinds of professionals. I don't understand why it wouldn't be equally useful in Software Engineering to have such a system.


> It would be really cool if the ACM, BCS, IEEE, or any one of those professional associations,

Not software engineering, but AWS/GCP/Azure* certifications are basically this. They are rigorous enough that you can't just luck into passing the tests, you actually have to study (for the higher level ones at least). And some are tiered, so you have to get a requisite cert, wait, then take the test for the next level.

ABET has a professional engineering exam for software engineering, but nobody ever takes it. This is the same organization responsible for licensing other engineering disciplines. I find it interesting that so many software engineers denigrate the field for not being "real" engineering, but none of those people are taking the FE/PE exams to create that critical mass necessary to begin making it a requirement for employment.

Part of this might be ignorance, maybe people don't know that software engineering has a licensing body already. Or maybe most of us don't want to be forced to get a license for a job.

* I don't have Azure certs, so I'm assuming here.


AWS exams are absolutely cheatable, there are numerous exam dumps on udemy. You can have a high schooler collect all 9 certs without a single hour of commercial experience.


Interesting idea but I think the arena of comp-sci is far more diverse than a traditional craftsman.

I finished an engineering apprenticeship once and it said that I had reached a basic standard in engineering but that didn't mean I could work on anything after that. My skillset was very much railway traction and any deviation would have required significant training.

Same with development. If you have done the same old crap for 20 years, that doesn't automatically put you on par with a genuine skilled senior dev (you might not even be good enough to be a junior).

I do agree though that I think there should be trade qualifications for developers and although I would feel aggrevied about sitting exams for things I have done for 20 years, I couldn't argue that it would remove a lot of these screening exams and could be just "Required: Grade 5 .Net developer or something"


This is why most hiring is done through personal networks, where trusted people vouch for the abilities and experience of a candidate. The coding test stuff is a workaround for when that route isn't an option.


I believe, for some people, personal networking truly isn't an option. For example, I work remotely for U.S.-based companies while living in a rural area of Germany. There simply aren't any professionals around here who do things that are similar to what I do, and whenever I meet any people at all IRL who are vaguely software engineers, their companies' pay bands top out at maybe half my salary expectation. So I don't just "organically" meet people IRL who could usefully connect me to my next job.

My role at my company isn't outside-facing. I'm never in any meetings that involve vendors or customers or anything like that. And my company has been very clear, that if I want to do things like attend conferences etc., I'd have to do it on my own dime, and in my free time. Needless to say, the projects I work on are never open source.

So where does that leave me network-wise?


Asking for fizzbuzz or a simple closure/curry add function (basically `a => b => a + b`) is far different than coding assignments.

I did one "two-hour" coding assignment that took me an entire day (and I was by far the fastest according to the interviewer who later admitted they'd not done the task and it also took them an entire day when they tried).

I don't plan on doing such a thing again unless I'm compensated fair market rates.


> I don't plan on doing such a thing again unless I'm compensated fair market rates.

Was having drinks with a friend and a recruiter friend of his, and the recruiter began bragging about how her no-name, average-wage firm had the HARDEST coding challenge in town (not even close to true...) because it took candidates as long as 16 hours to complete. She then began regaling us with a candidate who applied, was supplied the challenge (before any in-person, or even phone) interviews. They rebuffed his request to have at least a phone interview with the hiring manager prior-to committing to such time, and then were aghast at his asking if they would be willing to pay for his time given the extreme time commitment necessary.

You know how hiring works for my chef friends? A paid shop-test, show up on an off-day and work some station(s) to demonstrate you can walk the walk. You know how hiring works for trade friends? A paid shop-test, they show up on an off-day and work on a discrete piece of work or some piece of a project. Suppose one can only assume that firm's operational principle is to hire the easily manipulated...

Personally, I never ask for more than a 2-hour commitment and these days have lowered it to less than 1-hour, providing a scaffolded project from which one could reasonably extrapolate, and typically on their time. To follow up, discuss what they did, why they did it, any differing opinions or approaches.

Perhaps I am lucky to not have an LA/NYC/SF sized hiring pool, though.


> I don't plan on doing such a thing again unless I'm compensated fair market rates.

That's entirely fair, IMO. Once you're past the "prove you can code at all" step, being compensated for doing work is legitimate.


That doesn't make logical sense. We're talking about senior level here, and HR should certainly be able to filter out fishy SENIOR-level candidates based on job history. Maybe the problem is too many degree and fake job mills in Antarctica or something?


> Do you find these kind of tests useful?

I’ve hired dozens of Sr. engineers and given coding tests to almost all of them (after a 2nd round interview, never earlier).

However, my tests are time-boxes (1hr) and are about greenfielding an app (like a ToDo app) to see how they approach a problem rather than programming knowledge.

> Why?

About 33-50% of devs that pass the two interviews fail the coding test. Of the ones that fail, they’re great at interviewing but probably oversold their skills. Some are really poor at time management. Some are really poor at simplifying problems. And some are really poor at following directions.

It’s possible that they were on teams that carried their weight or possible they were exaggerating their capabilities or prior responsibilities.

The opposite has happened too where I’ve been blown away by the code quality and approach.

But overall it’s been a very helpful datapoint in hiring.


This is a hypothetical, but how would you suggest engineers that have issues with simplifying problems, managing time or maybe don't realize they're overselling their skills work in a positive direction to correct some of these things?

I've noticed many companies will give technical feedback if you don't pass a round, but seems like there has to be some kind of path to help people like this. Always been curious to me that "ideal" practices for managing time, communicating technically / in a business context never seem to really be written down. I learned some of this (albeit badly) at a few internships, but if there'd been some source to read even if it was only the first 25% that would've been immensely helpful out of college.


> simplifying problems

Practice rewriting personal projects. Once you have a working app, try to make it even simpler, clearer, more efficient, etc.

> managing time

If you’re on a tight deadline do the easy stuff first and if you’re falling behind get help with the harder parts or ask the team to scale back the scope. But if you bite off more than you can chew you run the risk of having nothing to show for it.

If you’re building a proof of concept, then start with the hard parts to show the team that it’s possible and worth additional time to finish it off.

> overselling their skills

Be honest about your role in a project. Don’t hide the fact that another dev was doing the primary research or making the technical decisions. Just take credit for the things you did from start to finish, without help (because those people that helped might not be there at your next job).


>About 33-50% of devs that pass the two interviews fail the coding test.

I don't mean to sound insulting, but if your organization can't tell if someone knows what they are talking about after an interview, much less two interviews, what the heck is going on?


All you have to do to be indistinguishable from someone who knows what they are talking about in two interviews is to say the things that someone who knows what they are talking about would say, in the way that they would say them. This is obviously a nontrivial skill, since it involves memorizing a lot of declarative knowledge, but for extraverted people with good memories it can come more easily than actual coding, let alone debugging. An interviewee who biffs it slightly can be very hard to distinguish from an interviewee who knows so much more than the interviewer that they identify errors in the interviewer's knowledge. And at this point there's a cottage industry of books and YouTube videos specifically designed to help people fake this stuff.

Basically what it comes down to is that doing something and explaining it are different skills. True of riding a bicycle, true of fixing a car, true of programming. Karl Popper never discovered shit, Harold Bloom never wrote anything worth reading, and plenty of bestselling novelists are utterly terrible as creative writing instructors.


>All you have to do to be indistinguishable from someone who knows what they are talking about in two interviews is to say the things that someone who knows what they are talking about would say, in the way that they would say them.

Sounds like a supervillain or something. Maybe you're asking the wrong questions or maybe I have a knack for it, but the only thing I can't detect is motivation, which is probably more important than current ability. One of my worst hires was a guy who could answer all the trivia; sharp as a tack, but didn't have the discipline to do the work. He was always slacking and I couldn't count on him to get things done, even with loose deadlines.

I mean most hiring managers don't bother calling references either. That yields a lot of insight; those references were hand picked by the candidate. Maybe call their last employer, even if it wasn't listed as a reference.

One of my first questions is, "Tell me about your favorite project and why." If they answer in a sentence or two, even when prodded or show little or no enthusiasm, it's a pass. You have to like the work to be good at it. Ask some follow up questions to their answer. A phony can only go so deep.

Second question is, "Tell me about your least favorite project and why."


Maybe you're just better at interviewing people than I am. (That sounds sarcastic, but it's not.) I'll try to take your advice to heart if I'm interviewing someone for a job in the future.

It's true that if someone has memorized the details of a "least favorite project" from dailywtf.com, or a "favorite project" from some other dev blog, they won't be able to go very deep into it; and if they don't have at least deep declarative knowledge about the technologies involved they won't be able to invent anything plausible either. Deep declarative knowledge, though, is still very different from debugging ability or the ability to find simple solutions to complex problems.

The enthusiasm thing is discussed pretty often in all kinds of job interview advice contexts, not just programming. I think many people make an effort to project enthusiasm in that situation, and in some other social situations like dating. Hopefully you're filtering for real enthusiasm instead of ability to emote, but realistically it's probably a combination.


>Deep declarative knowledge, though, is still very different from debugging ability

90% of debugging is reading (and understanding) whatever exception was thrown. I can't count the times when a really good dev asks me for help and I ask, "well what does the error say?" Once they read it, they're like, "never mind."

If reading/understanding the error doesn't work, it really requires pretty deep knowledge of the application architecture and possibly auxiliary systems and infrastructure to figure it out. If your application is custom, then they can't have that knowledge coming in and can only get it with OJT. That extra 10% is really the hard part, but you really only need one or two people who can cover the whole team on that extra 10%, and the process will help improve the people who aren't as good at it.

>or the ability to find simple solutions to complex problems.

That's where the genius really is. Reading the first few chapters of Donald Knuth's first volume really blew my mind. Probably won't be able to discover that talent in any interview process.

>Hopefully you're filtering for real enthusiasm instead of ability to emote, but realistically it's probably a combination.

Yes, it's just a piece of the puzzle. Lack of enthusiasm is more a red flag than putting a lot of weight in enthusiasm itself.

Making the simple complicated is commonplace; making the complicated simple, awesomely simple, that’s creativity.

-Charles Mingus


Yeah, those are two great questions!

> the only thing I can't detect is motivation

Interesting! I feel highly attuned to that, but less confident in some other areas that sounds easier for you to evaluate.

Funny how that works.


They can talk-the-talk, but can they walk-the-walk?


Not parent commenter, but the first two are often designed to be rather trivial. No point wasting hours of the candidate/interviewer's time if the candidate just doesn't work out. We've found people who straight up lie on the resume (claim to use a technology then backpedal when asked about it). And there are the kinds of people who just don't show up or can't speak passable English. And there are some who have a history of making CRUD apps for gov and giant corporations, but can't tell when to use an Array or LinkedList even when allowed to google it.

The real coding test later is to actually see that the person can code. Normally it's about following instructions, isn't afraid to get their hands dirty, bangs their head on the wall thinking of a solution but actually tries something.

There are also lots of people who will build everything except what you ask them to. You tell them to build a To Do List app and they go straight to building logins, adding profile pictures, and then complain that the test was too short.

We also often list technical requirements in the job description. The tech stack is always some niche things. It's okay if you don't know it all (nobody really does) but you have to be able to work with the code once you're working on the code.

A lot of people list buzzwords for everything they watched a tutorial on. Someone may have 3 years doing something but are actually bad at it. Someone may have learned it last week and want to be given a chance to prove that they're fast learners. Great, if you've been working with that tech for 3 years, this should be muscle memory. If it's new to you, well, let's see your learning process.


1) People lie. I found out one person had someone listening in and chatting the answers to them.

2) The interviews get progressively more technical. We do a non-technical first round (some people don’t show up on time, aren’t great communicators, etc.), then have a technical interview about high level concepts followed by a technical test.

3) Efficiency. We can learn more from a 1hr coding challenge than a 1hr verbal interview. And it requires less involvement from the team.


What you're measuring as "bad time management" is likely to some degree people being frustrated and dismayed and giving up because an arbitrary timed test isn't fun or interesting or rewarding, when there are companies probably paying just as much as you with an easier interview process. So yeah, anecdata can be tricky.


What I mean is if we ask them to spend no more than 1hr on a ToDo app, I’ve seen people spend 1hr on the folder structure, importing a ton of packages, abstracting logic, etc. and don’t build the basic CRUD functions.

So you basically end up with a giant empty shell of a project.


Do you do the coding test as a take home exercise or something done in your office?


Take home, paid, self-regulated. Asked that they only spend 1 continuous hour (verified by initial and final GitHub commits).


Commits are not verifiable, any half-competent git user can google how to change committer and author dates. I used to work for a company which did these "self-governed" tests and most candidates who bothered with these spent more time than they actually were supposed to to get an edge.


Fair enough, but we ask the dev to send us an invite to the repo when they first create it, so then we ask them to Email us an hour later and we check the repo.


This industry has no professional standards or relevant certification - what does a senior developer mean ?

You say you have relevant experience - will you show them your work so they can review it ?

I have a friend that has >7 years of experience - which was spent on obsolete stack in huge corporate contexts - you put him on a new project to bootstrap a generic app and he's basically a junior/mid with some debugging experience and knowledge of SCRUM.

And this was made obvious to him by multiple interviews/tasks he had to do - after which he basically had to spend 6 months getting up to date to be able to work on modern projects.


> what does a senior developer mean ?

That is the crux of this discussion. My experience is the title senior developer varies from firm to firm depending on size, and on which area of software or web development you're in.

A senior developer could:

- Have managerial responsibilities

- Have project management responsibilities

- Be involved in sales as the engineering representative

- Be a programmer who can troubleshoot and solve problems no one else can touch

- Be able to architect systems

- Be able to reverse-architect legacy systems

- Have extensive algorithmic experience and write programs that take decades of knowledge to do

- Have broad expertise, able to oversee teams doing anything from frontend to devops and networking, and know enough to steer them

- Have narrow focused expertise and be one of the top industry experts in their field


> what does a senior developer mean ?

I'll give that a go:

* They can work a problem independently

* They can make newer developers around them better

* They can speak to the product owner and provide feedback on rough development requirements and cost


2 other major things I'd say are :

- They can design systems that last (functional, performant, don't need to be remade constantly)

- They can solve a wide range of technical problems (investigate performance, build a basic UI, add some ML)


Agreed. I don't accept that someone who has, as another commenter wrote: "done the same old crap for 20 years" is unlikely to be a senior engineer.

I'm one of those. I don't know the latest JS frameworks. Or the ones that came before. But I can spot the problems in the project (e.g. performance, authentication, authorization), the security holes in the design before it's built, and coach newer developers on how to solve them. And I can make the case upwards for time/funding and make business cases for work.


That really makes no sense at all. How transferrable is the experience of an embedded developer to web development ? Or game development to enterprise applications ? I've seen people stuck in very domain specific stuff/isolated on some small part of a huge project that doesn't really translate to much outside of being a junior developer.

In my professional career I've went from C++/3D graphics -> .NET -> Angular -> RoR -> React -> Django -> Flutter -> .NET Core. I've also used Python extensively for scripting and personal projects, Clojure on a large personal project, F# and Rust on minor ones, Node.js and PHP side gigs, etc.

Which is to say I've hopped a lot of stacks and technologies between jobs and side projects.

And without a doubt I can say that the ramp up time, even between similar stacks was non trivial. It took me a year to get "comfortable" with RoR even when I had >5 years of experience with ASP.NET MVC and a year with Django/Python.

You say you can give estimates for time and funding and yet you don't know the library landscape and what work needs to be done vs. what can be handled by the platform/libraries ?

And that's not even mentioning stuff like cloud providers - where there's so many cloud specific offerings you can glue together that vary wildly between providers.

You mentor people about best practices yet you don't have real world experience with the particular tech stack ? You haven't tested your ideas and how translatable they are at all and you're just assuming it translates - I've been bitten by this a few times.

So yeah I disagree, when I interviewed people I like to go over some code, ask some questions, get a feeling of how much actually relevant experience a person has and how much potential they have in transferring skills/learning - but there's no replacing relevant experience. And being interviewed like this gave me some insight into their expectations so I could judge if I'm a good fit or not.


> That really makes no sense at all. How transferrable is the experience of an embedded developer to web development ? Or game development to enterprise applications ?

Why would you even look at stuff that far outside your specialism? That's like saying because I'm a consultant orthopedic surgeon I can be a consultant psychiatrist. As your career progresses you will be more restricted in the fields you can enter without beginning a new career.

And yet even in these way-out examples you gave there will be transferrable experience. Data driven design from game development; memory efficiency approaches from embedded? (I know nothing about it). It won't make you senior because you're way outside your field, but that experience is valuable.

> You say you can give estimates for time and funding and yet you don't know the library landscape and what work needs to be done vs. what can be handled by the platform/libraries ?

How do you think a manager does it? They talk with their team. A senior developer is no different except having more technical expertise than many managers. You seem to take a very indivdualistic view of the role of senior developer.

> You mentor people about best practices yet you don't have real world experience with the particular tech stack ?

I said "JavaScript framework" you broadened that to "tech stack". I don't need to know the ins and outs of React hooks - there's a reason I have developers in my team who are programming that stuff and not me. I was writing vanilla JS, then KnockoutJS, then Angular 2 so the principles are familiar even if I'm not mid-level in my expertise with React 18.

Mentoring is a lot broader than "teaching how to program framework X". It's about developing skills, helping them figure out career progression, and advice and guidance. Tech is the least important part of mentoring - they can go get training for that. I detest the phrase "transferrable skills" but they are central to mentoring.

> Which is to say I've hopped a lot of stacks and technologies between jobs and side projects.

Yes. Why didn't you stick with something?


I'm just saying a lot of people specialize in stuff that's not very transferrable.

I disagree about Angular being transferrable to React, really they take very different approaches to frontend development and reading React code of a Angular dev that just started with React is full of non-idiomatic code (seen first hand when migrating one project from AngularJS to React and getting the existing team to start porting)

It seems to me like we disagree on the role of senior developer - in my view senior developer is still an individual contributor and not a manager - sure you support junior staff, but you're the guy they go for technical stuff, reviews, better approaches - not the other way around. What you're describing sounds more like tech lead/architect role to me.

> Yes. Why didn't you stick with something?

Options available at the time, boredom with projects I was working on, generally liking to learn new stuff. A mistake in retrospective, but I'm transitioning away from development to a position where the breadth is an advantage.


> what does a senior developer mean ?

This actually does mean something, but SV companies have been handing it out like bullshit for so long that most of us don't even recognize that, and reject what it means as absurd because it's so completely unrelated to how it's used in our industry.

Junior means <= 3 years of experience.

Intermediate means <= 6 years of experience.

Senior means <= 9 years of experience.

Staff means > 9 years experience

"But my buddy Frank is a senior engineer at a company with three people and he just got out of bootcamp six months ago"

No, your buddy Frank is a junior engineer who took a false title


I humbly disagree. Experience means nothing. The ranking is defined by how capable engineer is.

Do not get me wrong. Title != ranking either. There is no industry-wide agreement what senior engineer is. Frank might be a senior engineer in his company -- because because he proved in short time he can perform on the senior engineer level of that company.

Each of us has limits. For some it is Senior, for some it even could be Junior. If people are happy in their level, there is no reason why they cannot be productive for many, many years.


>Experience means nothing

I disagree with this: In my 20 years of Software Engineering, I've seen plenty of super smart, knowledgeable and enthusiastic Junior and mid level engineers making wrong decisions that "in theory " look right because they just haven't seen enough production systems.

Battle scars are quite valuable, particularly when shit hits the fan and you gotta draw from your previous experience to understand what REALLY is happening in very short time.

As a specific example, in a previous startup 8 years ago, we were 3 engineers. Two of us had good field experience and the other guy had outstanding academic/algorithmic skills (he won several algorithm competitions). At some point some crap happened to production systems. When the 3 of us saw the logs, he suggested a couple of root causes that just were not really probable. The other engineer and I could "read between the log lines" and got to the real cause pretty fast.


> I humbly disagree. Experience means nothing.

Disagreeing with social norms doesn't strike me as humble.

I didn't say anything about whether experience was important. What I said was "these words are used to describe years of experience, and people trying to use them to describe skillset are making an error."

Most of the resume is about skillset. Why try to make this about skillset too? You've got entire lists of your specific skills, plus your jobs where and how they were applied.

To me, this seems like saying "no, college degree means I have the skills, skills are more relevant than education, I didn't actually get the degree but I have the skills, so I can wear the title"

Except that phrase doesn't mean "skilled," it means "years of experience," just like a degree doesn't mean skilled.

The reason we in hiring want to know how many years someone has under their belt is:

1) Everyone who's been in the job two years thinks they're the smartest and most motivated and that means they're just as good as everyone above them, and

2) Very little about seniority is actually about skillset. It's mostly about being able to define work, to be dependable, and to be able to clearly communicate expectations

.

> There is no industry-wide agreement what senior engineer is.

There doesn't need to be. This is nationwide, across industries, and a matter of law for government jobs. Agreement, by industry or individual, isn't relevant.

This would be like asserting that there's no industry-wide agreement on what age of employment is. There doesn't need to be; we have society for that.

You can crack any HR college textbook and look this up. These words have meanings, regardless of whether you imagine an industry wide agreement exists.

What use does a word like "senior" have if it means something different company to company? What's it even there for, at that point? In hiring someone, what do I learn from "oh well they performed at that company's senior level?" Do you think I know what every company individually defines that word to mean? Who is gaining calibration there?


> Disagreeing with social norms doesn't strike me as humble.

we're leaving in different social circles... so it is totally possible that the norm i see differs from yours.

I worked once for the startup where idea was, "engineer" does not need prefix. Salaries, responsibilities, you name it -- were different. The title (in R&D) was not. Nobody cares.

I did not say "skill". I do not care about skills any more than I care about years of experience. The only thing I care about is ability, willingness, and readiness to contribute. The same person might match differently in different companies. E.g., intimate knowledge of C++ in embedded systems (e.g., avionics software) might be highly irrelevant in web-oriented startup, which is pushing Python. Would 10+ years engineer qualify for "staff" position? definitely no. "senior"? -- maybe. If Boing had layoff, it might be an option for somebody to restart carrier. A good person will quickly grow through the ladder, in the same company or moving on. But immediately? please.

> a matter of law for government jobs One can think of government as a single employer. In my company there is a levelling guide. We do not care about government, government does not care about us. :-)


This assumes that experience is more important than skills. An extremely smart and motivated programmer with 3 years of experience is going to be far above a programmer with average intelligence who phones it in.


> This assumes that experience is more important than skills.

No, it doesn't.

It just means that the word "senior" measures experience, not skills, and is frequently misused.

It's fine to prefer a super skilled junior, and smart companies do that all the time. But that doesn't somehow make them senior, and failing to understand the distinction is a significant problem for hiring.

.

> An extremely smart and motivated programmer with 3 years of experience is going to be far above a programmer with average intelligence who phones it in.

My experience in industry and my personal viewpoint is that actually lazy programmers tend to be the best. They don't over-engineer things, and they write their tests, because they know it's better to spend an hour now than five hours next month.

My experience is that smart and motivated people tend to fall on the Knuth side, and not the McIlroy side, of More Shell Less Egg, and this is not what I personally want.

https://leancrew.com/all-this/2011/12/more-shell-less-egg/

Anyway, this is all irrelevant in my opinion. We can hem and haw until the cows come home about who's going to be a good programmer, what styles we prefer, what candidates we prefer, and who knows, maybe one of us will get lucky and say something correct one day. Most people never do. :)

In the meantime, these words formally refer to years of experience, and if you want to focus on their skillset, then read their list of skills, instead. The standard resume has not just an explicit list of skills but also how they were used in previous jobs, and this doesn't come up this way at all outside of hiring. It's a concreted matter; you can look it up in HR textbooks, and in government jobs it's a matter of law.

Similarly, their education isn't about their skillset, but it's important, and it should be on the resume. (Originally that said "phone number" in the hope of being funny, but it came off as sarcastic.)

Yes, skills are very probably the most important thing you can find on a resume about a person you're going to hire, and almost the most important thing about them in general (behind honesty and workplace toxicity.) But also other things are important too, and not every single thing on the resume needs to be about skill level.

When I want to know years' experience, there are other reasons I want to know that, beyond their skillset. It'll have a lot to do with whether they know how to document, how to gather requirements, how to write plans, how to contact stakeholders, how to push back against bad decisions, how to communicate schedule slips, how to make themselves available.

Yes, skills matter a lot, but also, the stuff under years of experience do too, and there needs to be a way to communicate that.

If I had the magic staff-reader wand, and two candidates, and it said "super skilled junior" and "average senior," there are actually times where I would prefer the latter. Not a ton of them, but also not zero.

Don't co-opt everything towards skills, or you'll blind yourself towards other important topics.


Insulted? Never. Annoyed? Many times, yes. It really depends on the type of test. If they give me the "this should take 4 hours" project, I usually tell them to take a hike because that's very disrespectful of my time and the 4 hours is complete bullshit if one is trying to deliver a perfect result. Coding puzzles are extremely annoying, but I can usually get through them.

The best coding tests are those that demonstrate you know how to code applications relevant to the job; e.g. a UI component or an API endpoint that takes a parameter and body and returns a filtered result. Then the test is followed up with discussions around what else could be done with it in the real world (auth, performance enhancements, etc.). Seemingly this would be the easiest and most rational path for both interviewer and interviewee, so I'm not sure why the industry is still stuck on those stupid puzzle exams.

"We can't find any good devs". No, your hiring method sucks and you're eliminating good devs because of it.


> The best coding tests are those that demonstrate you know how to code applications relevant to the job

Exactly. I am extremely confused as to why puzzle exams are common. Why can't the exams cover things that normal developers work through all the time? Things like reading existing code, improving error handling, debugging, basic performance optimization (e.g. don't do unnecessary work inside the for-loop), refactoring, etc.

When I was an interviewer I gave some code samples to a candidate. The code would have all sorts of problems in it and I asked how the code could be improved. If the candidate couldn't identify basic problems then it was a red flag.

For example, I made the code catch all exception types and ignore them. If you're a developer and can't tell me "You probably shouldn't do that" then you're not ready for production work. When I ask, "What can be done to improve it?" the minimum I expect is "Log the error" or "Don't ignore the error. Propagate it to the caller" or "catch only specific exceptions".

Funny enough, I once had a boss who wouldn't make the cut based on my example above. We begged him to stop committing code.


I agree, code puzzles suck. Our take-home assignment takes about five hours and we provide you with the skeleton project with a few schemas already there for you to play with.

https://github.com/play-oxygen/takehome-fullstack-engineer

What you do in this take home assignment directly applies to what you're going to be doing full-time while working with us.

However these kinds of take-home assignments only work if you have technical people on your team. If your founding team is non-technical and you are using an outsource shop YMMV. https://sergiotapia.com/your-interview-process-sucks-you-are...


No offense, but that still seems excessive. I'm guessing you still have another 1-2 hours minimum of interview time talking to the candidate afterwards too right? I would turn it down.

My FAANG interview was a total of 5 hours and it included mostly behavioral questions and two 45-minute coding exercises, both relevant to the job. I think that was a decent compromise, but even that felt heavy.

There are some exceptions. A 5 hour take-home project might be acceptable if you pay the candidate. For example DuckDuckGo paid me for their take home project, although it took far less than 5 hours to do. I appreciated that.

Still, with the developer shortage, I'm far less inclined to deal with these silly experiments when there's a handful of other companies that have more efficient hiring processes I can apply to.


I am coding for 40+ years...

My experience: * out of college people remember theory, cannot code. * mid-level engineers start to forget theory, can code, begin system-wide design. * senior engineers can design, forget how to code.

If the position of senior engineer expects hands-on coding (mine does), coding exercise during interview is quite revealing (sadly). Personally I do not believe in engineers who cannot code.


A senior engineer may be super rusty when it comes to programming but should be able to get up to speed relatively quickly. I'd be more concerned with their communication skills, their understanding of crucial concepts, and how they approach very difficult problems.


It could be the case.

The question is, how to distinguish one who can get up to speed quickly from one who cannot.

In the end, there are expectations of the engineer, and expectations of the company. If the candidate is super-rusty, question is, why? For extended period of time this engineer did not code, and apparently was comfortable with it. Would such a person be happy in the environment, where 50% of the time goes to direct development (remaining 50% -- mentoring, project management, etc)? Note, I wrote development, not coding. But writing technical docs is not development either.


I understand what you are saying. I can see using a coding test as a signal but it would probably be foolish to use it as a filter.

There are lots of critical skills and qualities for which there are no simple tests. For example, an excellent mentor could level up all those around them and provide far more value than a great programmer with weak interpersonal skills. In the end, it's a gamble.


It is so pleasant and so rare to agree with people these days. I agree with you.

It is signal for me. My interview consists of many areas beyond coding. But coding is one I explore too. (like write a methods with two sequential loops and single if in your favorite language.) You would not believe, some 30% people fail.

Mentor or programmer -- we need both qualities in the end. both extremes are harmful. To be a good mentor, one need to have respect of colleagues -- and it comes from contribution too.

gamble -- agree again.


as a multi-decade coder here, I believe this has stereotype-sort-of-truth, but is too restrictive in the labels.. look at musicians and think yourself a bit about how different kinds of musicians mature, and their interactions


Very true. I totally agree with you. :-)

I tried to highlight most common problem I see during interview, not judge specific people, or even less people category.


That doesn't clarify anything to me but I am curious, how do musicians mature / their interactions change?


I'm a Senior Dev. I dabbled in some coding in Jr. High, but go more involved in my early 20's. I'm now 60.

I've interviewed many engineers. My process involves asking open-ended question about important topics, and let them explain.

Write down about 20 or 30 tidbits about the language you expect them to know. Include a range of simple, medium, hard, and, hard-core expert stuff. See where they fail, or where they struggle to explain or discuss.

You can't "practice test" your way into that, and it's always obvious where someone is knowledgable, and where they struggle, or don't know anything.

Further, I ask probe them for their 'problem solving' approaches. How do you learn new things? Name some of your favorite books on programming/development/design.

Oh, that last one, that may date me, cause I come from a time when they had these things called books. We would buy them, and read them. I've found so many developers that just say "oh, I google for it". Sure, that's great, and I google the crap out of things also.. but, here is the fundamental issue with "learning" that way.

"You only get the answer to the question you knew to ask."

The other answer, the one that says "oh, there is a totally different, but superior approach" you learn from reading and getting a solid background on the capabilities of the language/library/module/environment. Books (and online courses or comprehensive material is equivalent) teach you that stuff.

The best programmer I ever hired read books. She (yes, she) was far younger, and unlike her peers, had discovered that reading and keeping some books in your library is valuable.

My advice, ask them what they read. If they "just google it", kick them to the curb. "Next".


You've described a solid method to interviewing.

I think your point about people who read books could be better described as people "intentionally learning" and using "self-directed learning". The sources of that learning are extensive and not limited to books or online courses. You could learn from a mentor, by trying out things yourself, by reading source code of open source projects, reading programming blogs, etc.

The underlying theme is that the person is intentionally trying to learn and doesn't require hand-holding or pressure to do it.


I agree with you @justin_oaks. I overly simplified that point. My main point is that learning should be as comprehensive as possible. Learn what your tools can do, then apply that to the problem, realizing that multiple approaches are possible. Know the tools well. It does not matter so much how you get there.

I've worked with engineers that do things a certain way because this is what they learned at some point, or saw, or googled. I show them "did you know you could just do this", and they are blown away. I too am blown away that they don't even know the capabilities of a language they have been using for years in some cases.


One thing that I always thought would really help, which "senior developers" as a group seem to be largely against, would be a basic competency test like the medical board exam or law bar exam. Do it once, and demonstrate that you can know some minimum standard amount of knowledge and can do the bare minimum of software development. It wouldn't test every development skill out there. It wouldn't test particular language competency. It wouldn't test for business domain knowledge. And it would not replace the interview or prove that you were the right fit for any particular job. But it could weed out 90% of the fakers who literally cannot program or know nothing about software that all the hiring managers in these threads are complaining about. It would help to prevent our current mess where every company in the world has to waste time doing FizzBuzz again and again for every candidate.

I don't understand why we are against this basic, minimal gate. We have this unreasonable fear of regulation and gatekeeping in our profession, which is ultimately to everyone's collective detriment. I'm not calling for 20,000 pages of federal regulations on software engineering. A minimal gate. Nobody benefits from the current world where anyone with no knowledge or skills can simply claim to be a software engineer. Nobody except the fakers.


A certification scheme like this could work, but it would need to be regularly renewed (every 5 years?). Passing it a single time early in one’s career is no indication that coding proficiency still exists later in one’s career.

In my experience, basic coding skills can really atrophy in senior developers who quit writing code years ago and are full-time architects/managers. I’ve interviewed more than a handful of highly accomplished people with decades of experience as developers and then engineering managers who couldn’t answer extremely basic coding questions (I’m not talking about Leetcode-esque BS, I mean stuff closer to FizzBuzz).


I kind of like the system they have for pilots: 1. A major exam/hurdle at the very beginning (FAA checkride), and then 2. Basic proficiency checks every 24 months done by an instructor. Vary that period as appropriate for developers. You get the initial weed-out of people who have no business being developers, and then periodic checkpoints to make sure atrophy is not a concern.


While I'm also for some kind of standard competency, if it were an exam it would create a market for test-prep books that cover just the material and nothing else.


that market already exists


https://news.ycombinator.com/item?id=27662804


As a senior engineer I have a very comfortable good paying job. When a tech recruiter or a job listing entices me to an interview they have a lot to overcome to get me to switch jobs. The recruiters especially often frame this as "Company x is so excited to talk to me!" and that I am going to be a great fit.

At this point it seems like the company wants me more than I want them. Then they give me a coding test and they have completely sunk any of the inertia needed to get me to leave my current job. Add to this hours and hours of interviews and I am like "Nope, I will keep my cushy job. Thank you."

It's even worse when these companies make me do a coding test before even talking to me. Companies need to get me interested and excited to work for them before I am going to spend my valuable time jumping through their hoops.


I have seen candidates present as a senior dev on paper, but clearly aren't senior dev based on interview performance or code. Without testing the code, there is a risk of hiring a senior dev (and passing on a more qualified one).

The disconnect between what is done day to day and the types of things asked for in an interview is because some candidates may refuse to do what could be seen as work product for the hiring company. There are companies out there that have taken this further and give a long "interview" that is actually work product. Making sure that the code being written is clearly demonstration / contrived examples reduces that potential conflict.

So yes, I do find coding tests useful as it allows me to put my best foot forward as a candidate demonstrating that I can write clean and well crafted code. Likewise, on the hiring side of the table, coding tests can help separate the candidates who don't care about the interview or will be a net negative after hiring from the ones that are willing to put in some effort.

Excessively long code projects are to be discouraged though - there is too much time that can be consumed by them and it biases to people who have an excess of time available to work on them.


Actually I'm also insulted by coding exams, and even more so by the entire hiring process. Anyone who has worked two or three decades in our field like you and I have knows that the key skill needed today is the ability to learn quickly. In our information rich field, knowing how to develop a solution to a particular problem often reduces to knowing how to search and utilize information effectively. Effectiveness comes from experience and isn't easily measured by assessing our knowledge of this or that programming language or algorithm. Current hiring practices are often mediocre and managed by people with limited understanding of the nature of what we do as coders. You are right in feeling that you should avoid working for these kinds of places.


Nobody who really works on software works on these kind of bugs.

That's kind of the point. Coding tests are minimal test cases. They're the sort of thing you wish you'd get when someone reports a problem - a distilled reproducible example of something so you can focus on the problem and only the problem.

That's how coding tests should be. Tests with huge blocks of "real world" code that doesn't add anything to the problem you need to solve are much worse.

no programmer is going to write anything useful for you

This is also kind of the point. Coding tests should not produce useful code. They're designed to assess someone's ability without asking them to do proper work. If you end up doing real work in a test situation without getting properly paid for it then you're being scammed.

The problem is often not the test itself, but the failure to do the groundwork to explain why I should take the test. I have a policy that I will never take a coding test before talking to someone from the company - I don't want to waste a few hours of my time on a role I wouldn't accept even if it was offered. I need to talk to the company just to sound out whether the role sounds appropriate, whether the company sounds like somewhere I want to join, and whether or not they're willing to put up with me. If I sound like a reasonable prospect from my resume then the company should be willing to invest 30 minutes on a Zoom call for me to answer their basic questions, and for them answer a few questions I might have too, before I commit to a more indepth series of interviews. If they won't do that then they're not serious about employing me...


"Coding tests should not produce useful code. They're designed to assess someone's ability without asking them to do proper work. If you end up doing real work in a test situation without getting properly paid for it then you're being scammed."

You can pass coding tests without being able to do the work. I really wished that a person who can do leetcode well can solve all engineering problems. Unfortunately, someone who leetcodes well may not be able to do the job.


No idea why you're downvoted - these points all make sense:

-I should speak to someone at the company before having to write code

-Interviews go both ways

-Coding problems should be appropriately sized, encapsulated, and reproducible

-Coding problems should not be real work


Downvoted because it's true :)


As a hiring manager, they are very useful.

As an applicant, you should consider it a good thing. You work in one of the few fields where it is possible to easily and quickly test whether someone is incompetent. If it weren’t for the tests you would be in direct competition with legions of bullshitters.


There's a difference between tests that use programming puzzles and ones that check for basic programming skills. I think the OP is opposed to the puzzle kind that have no bearing actual work.

Consider if you're trying to see if someone plays basketball well. But instead of asking them to show basketball skills, you ask them to jump hurdles and juggle. Sure, the people who can do those well probably have good agility and coordination that's necessary for basketball, but you're probably filtering out a lot of great basketball players.


The benefit of the coding puzzle platforms is as a hiring firm you get shown data about all applicants who took that test, and it’s standardised.

You also get pretty high resolution data about how your applicant performed (like their intermediate submissions, editing history etc) and how their solution performs compared to other applicants.

So all of that comparative info on a less directly relevant test can actually be more useful than n=2 or 3 on a direct proxy of your actual job.

Also I disagree with the premise that these exercises don’t carry over. The ability to puzzle through an algorithm you haven’t seen before in under an hour is directly useful and a common task in any domain that is algorithmically complex and not just gluing CRUD frameworks together (and even that gets complex once the scale is big enough)


> The benefit of the coding puzzle platforms is as a hiring firm you get shown data about all applicants who took that test, and it’s standardised.

You're arguing that standardized testing is a good indicator of candidate outcome. If this was the case, Universities would not be ditching the SATs en masse.

When you test for programming puzzles, you're just filtering out people who have grinded leetcode, just like how the SATs filtered out people who took SAT prep classes. If that's your goal, great. However, we shouldn't pretend programming puzzles are a good way to filter for great engineers. I can guarantee you that leetcode is filtering out a lot of great engineers.


Universities are ditching the SAT en masse because it is an excellent predictor of outcome, and that is interfering with affirmative action political objectives (much easier to get away with racially discriminating against more-qualified Asian applicants if you can deny knowing they were more qualified).

My goal in running a team is performance rather than political signalling, so I prefer to use the best performance metrics available to me.


As a candidate, coding exams are really useful for identifying companies I don't want to work for.

As a hirer, they're useful for not very much at all. They don't tell you if someone can produce working software, they don't give any insight into how someone thinks or how quickly they can learn, and they will rule out many candidates who are better than most of those they rule in.

They might help you whittle very large candidate lists down to smaller ones cheaply but if you have that problem you should be using a different approach.

With code exams as part of your hiring process over time you will skew your hiring towards presentees who have crammed and rote learned and are there for the wrong reasons.

Hire people you think are smart, try very hard to stamp on your unconscious biases, aim for a broad variety of experience levels and types in your team, give them all the support and help you can and make sure that you quickly fire people who turn out to have misled you about their skills.


Senior dev here, yes these tests are pretty useless for experienced hires. They only test a fraction of the skills a senior dev should have. Senior developers don't work on problems that can be defined in a paragraph, coded and finished in a few hours. Their projects will often involve a very large, complex, aging system sometimes with documentation and tests. They are given assignments immediately, have to dig into code and data with little help or supervision. Find the spot in these million lines of code where I can add 10 more lines. Their experience allows them to learn large code bases, manage risk, fill gaps in knowledge, proactively communicate with others and much more.

For interviews I've done code or design walkthroughs with the candidate. Have them open up their code and talk about it. And also I'll show some of our code, and have them read and explain it. Lots of good questions and discussion will come out of it.


This will probably make me come off as a diva, but I don't want to interview for companies that don't want to hire me for my abilities. I'm not interested in being just another index in the developers array. If that is what you are looking for, then you are quite bluntly looking for someone else.

I have an internet search engine on my resume. I have implemented a C compiler. Don't give me FizzBuzz.


How do you demonstrate to a complete stranger that you just didn't make up the resume, or wasn't pure dead weight in the teams who did the work?


There is no team. I did these things alone, for fun.

If you doubt me, ask me any question about them, any implementation detail, I can explain it to you. I can discuss parsers, stack frame optimization and calling conventions for days, weird edge cases in the C syntax, trigraphs, whatever; I can tell you not only how to implement a search engine index in minute detail, methods for compressing a lexicon, but how to implement a crawler, how to extract keywords from a website, how to identify the language of a website, we can discuss ranking algorithms, we can discuss absurd number of different ways a web server may tell you which character encoding a page has, I can tell you why some websites have the DTD "-//SoftQuad Software//DTD HoTMetaL PRO 6.0::19990601::extensions to HTML 4.0//EN", we can discuss the absurdly inconsistent usage of the lang tag, we can talk about how sequential writes sometimes are still faster on SSDs, we can talk about how to make a SQL query perform well on a table with half a billion rows, we can talk about blackhat SEO, we can talk about how to implement a hash table on disk and when that is and isn't a good idea, we can discuss B-trees.

I also have the finished product to show. It's not open source, but if all this fails to convince you, I can bring my laptop and we can walk through the actual code.

If all of this is bullshit, I'm the greatest bullshit-artist alive, and you can just put me in sales instead.


Sounds like a proper egomaniac and possible bullshit-artist. The remaining question is, how do you react if you are hired and asked to do something below your level?


I do work on all levels, that's the nature of working independently on large projects. If all the work you have is below my level, though, then clearly I'm not a good fit for the role and the role is also not a good fit for me.


I agree that refusing to write a trivial program at an interview is the best way to prove that you are a cooperative person and ready work at all levels.


If you want a guy to teach to jump through hoops on command, to bark and roll over, then I'm just not a good candidate for the job. You have failed my interview as much as I have failed yours at that point.

Look, I have been doing this for longer than some junior developers have been alive, and I have more than enough financial stability to be able to turn down offers. When I look for a job, I'm not looking for a way to pay the next month's bills or a means to get rich, I'm looking for somewhere I can make a meaningful difference.


You're not really convincing me, but I have been doing this for some time too. There have been a bit too many times when someone has been "working independently on large project" that turned out to be a total dumpster fire and I have had to clean up the mess. The hero programmers behind these projects all thought they were making a meaningful difference.


Exactly! This is the meat of it. If anyone wants to talk about what I have written, we can look at and run it together. It's all available to look at, discuss, and actually use. Isn't that more important than anything I could tell you with a test? Arguably, looking at my existing products, in depth, is the best possible interview.


I would feel uncomfortable enough about being shown proprietary code that I would end the interview right there. Not worth the legal risk.


Well I mean it's my IP so I can open source the particular file if it makes you feel more comfortable. It's not closed source because it's secret, but because it's harmful in the wrong hands.

Web crawlers are harmful to independent web crawling. It can be very annoying for webmasters when misconfigured, and abuse leads to honest crawlers encountering obstacles. It would probably spell the end of independent web crawling if every tom, dick and harry with a few spare raspberry pis set up their own search engine operation.


>Is anyone else insulted by coding exams?

Yes, I refuse to take them. Fortunately I've built a pretty good network and reputation over the years so I get hired over a lunch meeting. I haven't done a serious cold code interview in years.

The last one I did was around 10 years ago and I kinda wanted to do iPhone app development full time. I had several apps on the Apple App Store. They were live/downloadable and the published by was obvious it was me. One was in the top 10 list of its category at the time. The company still wanted me to "write a quick Twitter client app by tomorrow." I did it and they kept asking for more. I finally told them I wasn't interested anymore. There wasn't an HR department or anything, it was a small startup with funding. That company is out of business now. I'd have to be pretty desperate to do that again.

I've been on both sides of the hiring table. I suspect most good senior devs won't put up with these type of shenanigans, so but requiring this sort of stuff, you're limiting your candidate pool to poor or mediocre or rising star that will leave in a couple of years at best. You should think of a question that any senior developer would know but someone who's never coded wouldn't. Something really simple like, "when declaring a method, what keyword do you use to denote that it will return nothing?"

I say this every time this comes up, but the cargo cult is strong.


It depends on the exam. I would be at least somewhat insulted at this point to deal with l33tcode, brain teasers, and CS questions. A simple take home project can be okay depending on what it is. If it's a generic "JavaScript" quiz, I'd just walk out. Although this is tooting my own horn, I have enough of a background, enough references, and enough code on GitHub that the idea that I could somehow be lying to a company is more of a conspiracy theory than anything else.


I'm an individual contributor, but I've helped conduct quite a few interviews at my current place (where we have coding tests) and in previous places (where we haven't had them).

I totally understand where you're coming from. The coding test I had to do at my current place took me few hours to do. The company is great, and I'm happy at the place, however, I would be lying if I didn't think twice when I got told I'd have to do a coding test after the initial screening. Certainly, I did feel insulted. The reason I ended up taking it was because it was very obviously just a test (i.e. they weren't trying to get free labour out of me). In the end, it all worked out, and I'm glad I did end up doing it.

On the flip side, sadly I've worked with a lot of people who just don't know how to code. Coding exercising aren't perfect either, but it certainly weeds out a whole lot of bad hires. I've worked with individuals who have essentially become "the documentation person". This is fine if you're in an enterprise, but in a small 3/4 person team, that one person holds back the rest a lot.

I think having a _set_ coding exercise that doesn't get free labour out of the candidate, it can help a lot towards having a stronger team. As a candidate, do be careful whose coding exercise you end up doing. There are still quite a few companies that try to get free labour out of candidates.

So yes, I think it is insulting unfortunately, but it is a necessary evil to build better teams, and avoid those who are, quite frankly, lying and are going to hold the team back.


> The coding test I had to do at my current place took me few hours to do.

> The reason I ended up taking it was because it was very obviously just a test (i.e. they weren't trying to get free labour out of me).

But it still takes the same amount of my work.

Let's say I'm looking for a new job at a time when I still have an old one. I send out 30 resumes. I get 10 responses that want me to take a coding test, and each one takes me 4 hours. That's 40 hours, at a time when I already have a job, and while I'm still trying to have a life.

So, no. Just no. I won't take your multi-hour take-home coding test. I mean, if one of the places was my dream job, I might for that one. But if you're a run-of-the-mill employer? No.


I totally agree, and you should be selective with take home tests. I do very few and the potential role would really have to be worth my time if I were to get hired. I don't believe dream jobs exist. I see a jobs as a means to an end. It's a tool for me to live my life the way I want to live it.


Interviewing people sucks. It's so hard to gauge the competence of a person in 45 minutes. Good code tests are a decent, standardized way of getting a read on a person. They are more fair and suffer less from the "confidence = competence" fallacy.

My experience has been that good coding challenges correlate strongly with good team members. It absolutely filters out good candidates on occasion, but it also filters out ones that can't code too.

From my perspective as a candidate. I just pay attention to what the recruiter says their interview process will be, and brush up on relevant skills.


> Good code tests

> Good coding challenges

Now only if we can get a service which administers those "good" tests instead of testing using puzzles and algorithms.

You mention that you have experience with good coding challenges. Can you give examples of what is in such challenges? Perhaps the folks reading here could get a better idea of what a good challenge is like instead of a puzzle/algorithm test.


Cracking the Code interview-style questions are what we are currently using. I'm not sure if that's where they originated, but that's the style of question. I think the key at my current gig is that engineers who interview regularly do the interview challenges. This ensures that the questions are reasonable to ask, and that effective guidance can be given to candidates.

For those without large engineering staff, HackerRank is good, in my experience. I use it for practice. I've had bad experience with leetcode, we used it at another place I worked and some of the crowd sourced questions were wrong.


Ugh, so many bad practices being defended in the comments here.

I've been on both sides of the table:

- As an interviewee, my experience is that companies that use coding exams are not places I would want to work - companies that use these exams have a poor hiring process or any technical interviewers themselves either are not proficient with the technologies they are interviewing for or cannot discuss the technologies effectively. Any of those cases is a big red flag for me.

- As an interviewer, there are two problems with coding exams: either they are so short that they can only test CS knowledge instead of the actual required skill (which is translating the business problem domain into code) or they are so long they impose too great a burden on the interviewee. ("Oh, don't spend more than 4-8 hours on this project" is not an acceptable burden in my book.)

Further, the hiring manager is not the one that should be evaluating the technical ability of candidates. (Well, unless the hiring manager is hiring Employee #2 for a start-up, but that should be more about trust and hiring someone with a track-record of success.) The technical staff should be evaluating the technical capability of the candidates, and the technical staff doing the interviewing should have both the technical knowledge and communication ability to easily screen any candidate in 30-45 minutes of casual conversation, tops. The hiring manager should be looking for team fit, communication ability, and other soft skill abilities the candidates have.

Anecdotally the businesses I've encountered in the Upper Midwest that do actual interviews instead of coding exams all have solid developers as employees. The businesses that use coding exams have the second-rate (and lower) developers writing code.

No other field in engineering, barring positions requiring the professional engineer's exam, requires candidates to go through an engineering-specific exam. The employment contracts that are drawn up usually include a 90-day probationary period in which the hire can be fired again for non-performance. I don't see why software positions should be treated any differently, since that hiring process has worked (and worked well) for other engineering fields.


>The businesses that use coding exams have the second-rate (and lower) developers writing code.

So FAANG has second-rate developers?

I would say the opposite anywhere that does not test has second-rate developers as they let anyone who can talk the talk in.

E.g. I have seen senior 5+ years tenured engineers at non-FAANG companies with weak coding tests, write O(N^2) code in production; then I took a look at it and reduce it to O(N) and N was not small it was pushing 50,000 and was only expected to grow...


If coding has taught me just one thing its.... keep your ego in a box.

If you need a job, do the fucking exam. if you dont, tell them to pound sand. Its as simple as the dynamic of power.


> It makes me not want to work at places that do this.

I won't work places that do this. It's inconsiderate of my (highly valuable) time, and it makes me think that the company has trust issues.

You'll especially feel this way when you look at The hiring process for CTO[1], which is more conversational. I believe senior engineer interviews should be something like this, but with more heavy technical questions.

[1] - https://www.karllhughes.com/posts/cto-interview-questions


Personally, I find the algos interview to be the bullshit one. I've lost track of the number of people we've interviewed and hired that do well on it and then once they are working for us go ahead and design unnecessarily distributed systems (i.e. split some functionality across two or more services when there wasn't a scaling requirement that necessitated splitting things up) and thought nothing of all the extra CPU and memory that was now going to be used on serialization/deserialization between those two services and that's before you consider to all the added incidental complexity from (1) separate deploys, (2) separate on-call shifts, (3) separate logs and metrics, (4) more complicated migrations of interfaces between them, (5) increased complexity in testing that the services work correctly together, etc.

In the last 7.5 years in my current role, I have yet to apply much if anything I needed to study for the algos interview for this role. I get that some adjacent roles in the company might occasionally need this stuff, but it's used rarely enough that it's a better skillset to be able to find the algo you need when you need it then study to be able to conjure one on demand only to have that knowledge atrophy while on the job.


It's insulting when a company is the one to reach out to someone who is senior, and already employed, but then added to some pipeline where you are expected to first jump through hoops like technical quizzes. Demonstrate you've done "preparation" for interview, asked why you want to work for company X, etc.

Smaller companies shouldn't delude themselves they are some FAANG type company with huge comp that thousands of people want to work for.


The tests are close to worthless. It is easy to check if someone truly did what they claim on their resume by asking questions about projects they worked on. Caught way more people misrepresenting on their capabilities by doing that but you have to have people doing the interview who has both breadth and depth in multiple areas.

I rarely see interviewer(s) asking questions to try and discover the person level of knowledge but instead ask questions showing theirs.


We had a conversation about this a short while back in a local Slack channel, and my personal opinion is that I have never been fond of coding as an exercise for a technical interview, especially if it is a whiteboarding exercise in front of others or a timed event. Due to their simplicity, they offer little beyond basic syntax or entry level abilities if you're looking at a short timespan, and anything more complex is probably prohibitive if you're interviewing multiple candidates, because the code has to be reviewed.

Take home coding exercises are a bit better, but someone still has to review it, and who has that kind of time if they are interviewing a lot of potential candidates?

Instead, if a coding exercise is necessary, I would lean toward having the candidate perform a code review on a known bad piece of code, looking to see how they spot inefficiencies, and then have a separate technical conversation, akin to refining a project that someone would work on. It would be more real-world, and I think you could gauge their thinking process better, and still get an idea of the complex thought processes involved with senior development.


I find that coding tests are a huge red flag. I usually stop any conversation as soon as I know it will be required even for freelance side jobs.

As hiring manager I do not disagree with basic tests for junior roles but for senior roles it definitely does not assess where the value is, coding is only part of the equation. I believe that a short call with a peer from the team to review a merge request/architecture decision is far more interesting.


As a rule they aren't insulting – but as with anything there are good and bad implementations.

I've been on both sides of the fence, as both an applicant and the hiring manager. I personally hate "leetcode" nonsense, "find-the-bug"-style quizzes or algorithmic white board challenges. I'd consider going through with them as an applicant, depending on the organisation, but I don't think they're useful tools and it would be a bit of a red flag that suggested they didn't have a great hiring process. But I realise that people have different views on this and try not to take it personally.

We have settled on using a compact take-home coding exercise, which is a pretty representative example of the sort of thing we do day-to-day. There are obvious downsides to this, in that it requires a few hours of commitment, but we've tried to be as flexible and light-touch with it as possible. My experience is that roughly 50% of people totally flunk it – and this is post-first-interview, after we've had a decent chat and decided that they can communicate and have the right level of apparent expertise. And our analysis isn't that harsh – it's basically "did you write code that I could read and which does something in the ballpark of the spec". It's pretty surprising to see so many don't hit that bar.

I am very much aware of the requirement to be respectful of everyone's time, so I figure this is probably fairer than having people waste effort continuing with the process when they've no chance. Any approach you take is going to have a set of tradeoffs, and all you can really do is make sure that whatever process you put in place is checking for the things you want.


Senior developer here. I've been on both sides of the process.

Firstly, let me say that I hate the idea of take-home tests. I don't consider myself particularly woke, but it seems obvious that forcing people to find a two hour slot in their personal lives to produce their best code for a throwaway project, which may not even get them the job... that's going to discriminate against parents of small children and those caring for loved ones. Statistically speaking, females and BAME applicants will suffer more.

So if you feel you have to offer this, always allow a path for people to opt out (WITHOUT them giving you a reason. Just don't.) And as an applicant, if you feel this part of the process is going to be difficult for you due to your circumstances, always ask the question.

How they treat your request will be one of the best benchmarks you can get of whether the company's equality and compassion policies have teeth, or whether it's just lip service.

That said, for some people it does work. It makes it easier to determine a person's skill level. It weeds out those who can't code or don't have any experience of best practices.

So when I had to quickly find myself a new job recently, I was asked to do a couple of take-home projects. Knowing that the roles all had an element of knowledge transfer and mentoring, I set up a screen recorder (with audio) and narrated my solution.

It felt completely weird. It was totally unnecessary. But it meant I could watch it back with fresh eyes, and see whether I justified my decisions well, whether I explained myself cogently, and what I would do in the future.

So no, don't be insulted by the process. Use it to your advantage.


Hiring manager here-- we send out a coding assignment to do a small UI component.

It has its hits and misses.

Hits:

1. We suss out a lot of people who've been a senior engineer on their resume for years but aren't up to date on practices we care about, like accessibility, semantic HTML, app architecture etc. That stuff is important, I don't want to hire folks who ship apps, but the apps aren't WCAG 2.0 compliant or poorly tested.

2. It can definitely help rescue a candidate that we're on the fence about.

Misses:

1. Some candidates blow it off or phone it in, and out of fairness to those that did well, we can't advance them. It's a shame too, because often the reason they do that is because they have other offers on the table, or other companies they're further along on, not because they have an attitude.

2. We say it's a four hour assignment, but in truth people who do well take around 6-8. It's not because the work takes 6-8 hours, but people go above and beyond, you can tell from the git commits.

Overall, I like it our coding challenge and won't hire without it, but I'm not saying it's perfect.


If you sent me a "four hour assignment" in order to determine if I'm "up to date" on practices you care about, that is a *strong* signal that you think your time is more valuable than mine. You can be damn sure I would "phone it in" at best, and most likely you'd never hear from me again.

You wanting to focus on specific skills/knowledge is a fine goal, however if you really care about these things at the senior end, you should make the investment yourself as well and take the time to actually speak to a candidate. Who knows, you might just learn something new.


Probably "up to date" was the wrong phrase-- it's more "are the fundamentals there". I've had lots devs send me assignments that miss the table stakes of front-end dev work: is it keyboard navigable, are proper elements used, is the CSS something we can actually work with.

I'm not sure what you're suggesting? Just a chat where we go over all that stuff? I'm happy to take into consideration other ways of assessing.


Alternately, you crush the exam and negotiate hard because you know you've beaten everyone else. It would depend on what I want.


To be honest I've turned down roles paying an extra $20k because the interview process is so drawn out. I'd have to be reaaaaally enticed to spend potentially days interviewing.


I'm unwilling to give a prospective employer 4-8 hours of my life.

Asking people to do it is discriminatory, because there are definitely people who are in life situations who can not afford the time to work their full time job and fulfill their other life obligations and do your quite extensive test.

If you're not spending equal employee time to my time in the hiring process, it's a no from the start.


That's a-ok, I often tell candidates that if they can't, no worries and they're welcome to apply again later on down the line or if they no longer want to apply that it's absolutely fine as well. It's never really had as negative reaction , most people understand that you have to be assessed in some way. Also, I make sure to tell people that if you need more time it in no way negatively affects how we assess the assignment.


And if you were, say, interviewing a single mother who absolutely did not have the time or resources to take your test, you would effectively be removing her and all people like her from your candidate pool which is a nice way to discriminate against a class of people without leaving much evidence of doing so explicitly.


Not at all-- I'd absolutely change and flex my process to accommodate someone in that situation. I never said I wouldn't from my original comment, that was something you inferred. If you know a single mother who needs a great coding gig, feel free to paste a LinkedIn and I'll reach out.


Do you pay them for the 6-8 hours they put into it?

I know back in the day when I travelled to interview somewhere, I got paid the mileage I drove and sometimes the time I spent there. Just curious if that kind of thing still applies.


Company doesn't, but I absolutely would be open to that, I had that in some other industries I was in and it was appreciated. FWIW, I advocate for strong comp for whomever I hire.


> We say it's a four hour assignment, but in truth people who do well take around 6-8.

Speaking of insults. You are wasting people's time and treating them without respect.

I say this as an applicant who has also wasted an entire day on an outrageously complex take-home assignment "estimated" to only take a couple of hours. It left a bitter taste.

In my view, anybody who phones in one of your assignments, or walks away altogether, is dodging a bullet.


I've been there too, I've definitely had bad experiences with coding assignments. I try hard to not treat others as I have been in the past.

I run a respectful process. I offer people ample time. I tell them that asking for more time is a-ok. I answer questions.

But at the end of the day, some people just really knock it out of the park (maybe around 30%?), and I don't think it's unfair to reward that.


You consciously misrepresent how much time is required to do well.

You could run a fair competition by stating clear requirements. Instead, you reward those who are most willing to burn through extra personal time.

That is not respect.


I think I've nailed the source of this disconnect. I don't say it's a 4 hour assignment and then get mad when people don't spend longer, or filter our anyone who does well in the four hours. I go solely by the code I'm given. It's just that around 30% or applicants or so clearly spend longer because the assignment is so well done. That is all. I'm guessing this end result would happen with any amount of time I specify, be it 6, 9 or 20 hours.

Some people just really put effort in, and as I said in my top comment others don't. Often those others don't for valid reasons-- they have an offer on the table, they want another job more, etc. As I've said in other comments, I have no way of knowing who has put in more time (aside from Git history which isn't a clear picture, but indicative), nor would I create a malicious metric. Also, I clearly stated that I viewed this as a con of our hiring process. I'd love a clearer way to even out the time, but barring bringing in some more stressful process like some timed app or timed challenge, I don't think I can.

Say what you want, but rewarding folks who spent more time isn't disrespectful to other people who didn't. No job is a first come first serve scenario, you're both assessing each other on a variety of things, and it would be unfair of me to disregard people who excel out of fairness to those who don't.


> but aren't up to date on practices we care about, like accessibility, semantic HTML, app architecture etc. That stuff is important, I don't want to hire folks who ship apps, but the apps aren't WCAG 2.0 compliant or poorly tested.

If a seasoned dev hasn't been asked to do this in the past, i.e. it's not a requirement for delivery (it's the client or product manager that decides whether they want these features, not the senior engineer), then I'm fairly sure they'll get up to speed with these practices when they're asked to. I kinda feel you're probably binning some decent potential hires.


I think this comment might get more acrimony than my top-level one, but at the end of the day I'm not that interested in hiring someone who only learned good fundamentals because it was asked of them or they were paid for it.

I made a decision in my career to consciously level up my skills-- testing, accessibility, documentation, etc-- all without being asked. Having good fundamentals just demonstrates interest at the end of the day. I don't want a master craftsman here. I just want someone who cares and tries to write good code. That's not an unfair ask. I have to hire someone I want to work with and collaborate with, and truth be told if someone won't learn a best practice in our field with being paid or it being requested, despite how nice they are, I'm not excited to work with them.


> I'm not that interested in hiring someone who only learned good fundamentals because it was asked of them or they were paid for it.

I'm not talking about someone with "good fundamentals", I talking about experienced devs who can tackle any task asked of them. The sort of developer who can walk up to the docs and start working productively with new (to them) technology within a few days and produce high quality code. The sort of developer who's probably keeping one eye on developments or features (say in Azure or AWS) that might be useful even if there's no remit for them to exploit those technologies right now. That stuff is always knocking about the back of their heads.

You cannot possibly expect every developer, even very experienced ones, to know every nook and cranny of every technology you specifically work with, but they are experienced enough to know how learn new skills quickly to complete a project.

> I made a decision in my career to consciously level up my skills-- testing, accessibility, documentation, etc-- all without being asked

I do the same, probably a couple of hours most evenings, but with the best will in the world there isn't enough time to learn and level up on everything. And that's even after nearly 40 years in this game.


> You cannot possibly expect every developer, even very experienced ones, to know every nook and cranny of every technology you specifically work with, but they are experienced enough to know how learn new skills quickly to complete a project.

I am specifically not doing that. Nor did my comments really warrant thinking that (maybe because I mentioned WCAG 2.0?). Testing, accessibility, architecture and good semantic code are skills that apply to all work in the front end space. If you're an experienced dev but you don't do those things, then I'm not exactly sure what experienced means? I don't want someone who has just built features without those things. It doesn't matter what framework you use or what libraries you know. I'm not sure how you took my comments to mean I'm doing this. I specifically tell candidates that we can teach them our specific tooling and about our business domain.

From my original comment I said "...aren't up to date on practices we care about, like accessibility, semantic HTML, app architecture etc" Perhaps "up to date" was the phrase that got under people's skin? Best practices are always evolving, but in all those categories, there is a pretty commonly understood bar of skills, and I often receive assignments that miss those marks.

Finally, I have encountered situations where one of those has been very under-skilled (most commonly accessibility), but the others were strong (testing, architecture, html/css). I have made calls in that case to hire with a plan that we'll work on the lagging skill.

I think in this case my comments were extrapolated on in an unwarranted manner. The internet isn't a perfect way to communicate, but at the end of the day I didn't say anything that is unreasonable. Also, it sounds like we're both in agreement on what makes for a good engineer.


> I made a decision in my career to consciously level up my skills-- testing, accessibility, documentation, etc-- all without being asked.

Speaking of which... I'm intrigued by your test, in no small part because I'm not sure I'm up on accessibility (or much that's happened in the front-end world since 2015). Would you be willing to send it to me, along with any evaluation rubric you might have?

Email's in my profile.


6-8 hours? That’s a whole working day. I’d have to seriously weigh that up against an extra £500 in my bank account or whatever… I’ll happily do an hour or two, something I can fit into a lunch break or evening. But, for me, that’s way too long.


If you do this with just 200 applicants for various positions, you've wasted an entire man-year of time that would be better spent on other things.

Why would I put in over a thousand dollars worth of time into a "maybe"? Is your company going to pay me what I'm worth for that time? Is it worth my time to implement some useless widget?

Loads of other companies are great to work for and don't require this kind of worthless time investment.


This man hours comment is fair, and it merits consideration. Any assessment that doesn't result in a hire is wasted man hours. I'm not exactly in this profession to talk to other people.

Also, if you're making $1k for 4 hours, you definitely should not work for me, you have a good consulting gig going and I wouldn't be able to be competitive.


If I manage to get something done as described in 4 hours I will not work a minute more, I will not polish to code, I will not go above and beyond I showed I can do what is required and that should be all these tests do.

I demonstrate that I can program and that I am evidently not lying on my resume regarding basic requirements.

I do not spend more than whatever it takes me to finish the task or whatever amount of time I have slotted for the task, if you say your task takes 4 hours and I find it is poorly specced and there is dependency issues that have to be fixed meaning that nothing will build and it takes 4 hours to debug it all then you get not code at the end just a description of how the requirements you have written out have problems with it (happened one time), if I somehow miraculously manage to do what is required in 2 hours that's what I do. I have kids and other things to do.

If you are privileging people who spend 6-8 hours because their stuff is better than the people who spent the agreed upon 4 hours not only are you getting people to spend 6-8 hours of their time without compensating them, but also the people who spend 4 hours are probably completely wasting that time. I hope someone posts your company's details somewhere so I can avoid them if necessary. I'm much too busy to waste 4 hours like that. Either that or just say so up front so that people can say yeah, I'm currently not desperate enough to spend 6-8 hours working for free. But I guess you don't want to do that, do you?


I would encourage you to read my other responses, I think together they offer more perspective on how I run my process. I'm not privileging anyone, lots of people submit great work and I really have no way of determining who went above the time allotted. I don't track time or cut anyone off (how would I? It'd be some logistical nightmare or I'd have to use some weird app to track how long they code?).

I gave an honest assessment of what I feel are the pros and cons of a hiring process. You seem to have inferred a lot based on that comment.

Finally, I do occasionally get assignments that don't work. Why even bother doing it then? Those tend to confused me the most. Would you hire someone who sent you any work that didn't compile?


I’d rather smash out a four hour coding assignment than do a brain teaser at 8am, followed by another one.


And if you read the other comments, if you put in full 4 hours you'll be rejected because there are people who will put in 8. And the hiring manager won't spend more than 20 minutes on your submission.


Well, you can also get rejected doing a few brain teasers in a row. Sometimes I even feel dumber at the end because I didn't "see the twist" or whatever.

I've done a few of those take homes, and none of them ever really took me more than an hour (and were accepted). I have 50+ side projects using a few standard templates I've made for myself, so I can knock these out really fast and just show what I can build


no brain teasers, riddles, gotchas or whiteboarding in our process. And I make sure candidates know it so they aren't stressed about it.


> people who do well take around 6-8

Sounds to me like you've found a good way to filter out all but the most desperate candidates. Which, given your overall demeanor that suggests you're also a mandatory nights-and-weekends-with-no-extra-pay type of employer, is probably what you're looking for.


not all-- I'm proud to say I've never once had a team member pull any all-nighters. I think you're just reading too hard into my comment. I tried to give a thoughtful assessment of a stage of my hiring process, not cheat or lie.


It's not that I'm against a "4 hour test" per-se, but it's highly dependent on the context.

1) If that type of test is used early in the flow, then that is highly disrespectful. That type of time commitment should only be used at the end of the hiring funnel when everyone else has already signed off on me, and I have signed off on the company.

2) I need to know that the time commitment is symmetrical. While coding over zoom isn't perfect, at least I know the company has done the analysis to say "this candidate is worth 4 hours of our company's time". Once a hiring manager sets up a 4 hour calendar block on a senior engineer's calendar, you will immediately get pushback of "Is there anyway we can shorten this exercise to 1 hour? 45 minutes?"


1) The assignment is mid-flow after we're reasonably sure we'd want to work with said person, and if the assignment results in decline, I give actionable feedback why.

2) The time commitment can often be more on our end, several engineers review it, it's probably equal or more man hours. I think doing a block of fixed time would be more stressful, I've had to do that and it wasn't fun.


I'm surprised at the downvotes and disagreement for this. I'd be fine with a four-hour take-home. I got one of my jobs by doing that.

One four hour offline/asych event, in practical coding rather than a whiteboard brain-teaser, to gauge the fit of a multi-year job relationship, is perfectly reasonable. I think the sense of entitlement might be getting a bit too high if people here say they would blow that off.


It is an unequal time investment. I have had multiple take homes where I got ghosted afterwords. Maybe 100 people did it, candidate number 3 did a good enough job and they never even took a look at mine. At least with a phone call or an in person interview I am assured they have some skin in the game.


I don't ghost-- no matter when I decline, no matter the stage I write and send the candidate feedback. I've been ghosted, it sucks.


Why is an unequal time investment a problem? Plenty of things in any employer-employee relationship are going to be unequal.


The rest of the comment addresses why it is a problem.


I think the sense of entitlement is on the hiring manager's side if they want multiple applicants to burn through 6-8 hours of free time to take an assessment of their skills. Then those same hiring managers are the ones complaining about how they can't find employees.


No entitlement here. I copiously thank the candidate for doing it and make sure to offer feedback if it results in a decline. I would offer reimbursement if our company offered that (and I'll definitely bring that up).

But, I have to assess candidates somehow-- I don't do whiteboarding, riddles, brain teasers, etc. I wish a discussion would just yield the results I need, but as many other top level comments here state, that's just not the case.


How can you tell from GIT commits, sometimes people with busy life's only get short periods of time every night and might have weird commit times.


That's an interesting one, I can't tell fully as the commit doesn't state when the coding started, but you can just take a look at the history and see short bursts of work. Generally it seems like people do the assignment in 2 bursts of work.


I think it depends, I have been coding for about 15 years, shipped a lot of Java applications, a lot of Rails, shipped a react system so some Javascript, and done some Python scripting. I applied for a job that said it was heavy OO Python, I said I had some some scripting but hadn't done much large scale Python. I didn't quite complete the coding task as I didn't realize they wanted Python 3 and I was having a few weird problems as I was used to Python 2. If they needed somebody fluent in Python/Django/Flask from day 1 and didn't have room for me to learn then I wasn't the right candidate. If they did and wanted my other experience in the domain area then this was a pointless box check test that I clearly communicated I may not do well on. Funny enough that position is still open a year later and I am doing the same job at a different company in Python/Django. I said in the interview I would need time to learn and they said it wasn't a problem.


Just curious, has anyone ever had a candidate actually fail FizzBuzz in front of them? I think the original statement was something like 90% of candidates can't do it. In my experience, every single candidate I have ever given that question succeeds. When I am asked FizzBuzz on an interview, I am actually insulted.


Not FizzBuzz, and not me personally, but from someone I worked with and I have 0 reason to believe is lying:

After a few disappointing responses to better questions, interviewer decides to give a "freebie" and asks for a function that determines whether an integer is odd or even.

Candidate produces something that loops from 0 to the provided integer, incrementing two separate counters, then comparing them at the end.

Interviewer manages to stammer out something like "are you sure there isn't a more efficient way to solve this?"

Guy furrows his brow for a few minutes more, then a light bulb of sorts goes on, and he declares "Of course! ..."

"... I only need one counter!"


That seems more like a failure of mathematical reasoning than in programming.


I would be amazed to meet someone who, if asked whether 3042041 was odd or even, would follow this exponential-time procedure manually. Unless the candidate was such a person, it's a failure of programming.


They just seem numerically illiterate to the point of not realizing how to determine odds or evens.

Though perhaps, if they were more competent as a programmer, they would skip decimal math altogether and use the ol' bitwise operator trick.


FWIW, gcc 10 generates identical code for `i % 2 != 0` and `i & 1 != 0` at -O1.


Yeah, but "doesn't know about the % operator" is a lot less damning than "can only think of an exponential-time algorithm to determine if a number is even". Like, this is terrible, but it's linear time:

    char s[32];
    sprintf(s, "%d", n);
    char c = s[strlen(s) - 1];
    return c == '0' || c == '2' || c == '4' || c == '6' || c == '8';


Not fizzbuzz, but at one of my past companies we did a similar low complexity problem (use any language you fancy, google if you need to, you get a list of things, filter them by some attribute and group it by another, print out the list per group) during a phone screen, and people did fail it. Typically doesn't happen in later interviews, unless people have a complete mental breakdown due to stress.


I dunno.

Personally I do well with whiteboard interviews and similar things most of the time, both in terms of being good at the subject and in terms of making up for my deficiencies in terms of bluff and bluster (I like to joke that there are two answers to any interview question: "look it up in the hash table" and "look it up in the literature")

For all the talk that "hiring is broken" and that managers and candidates need to do something about it there is remarkably little talk about what to do once you've hired them (other than occasional threads that are fashionable from time to time such as "what to say in a 1-1 meeting when there isn't really anything to talk about") If the situation of vetting candidates for quality is so bad you might think labor unions for software devs would be good for management.


I'm not insulted by coding tests. Not even simple bug fixing - those silly bugs may not be production bugs, but I cannot be the only one who makes mistakes on the fly as I code, then self-correct immediately afterwards. How many times a day to you self-correct something in your code? Stupid tests for bugs show you can do so, so I'm not insulted by them.

What I am insulted by is the leap from "have a conversation" to "go take these tests". I want that conversation to know if I am even interested. It is not respectful of my time to make me jump through hoops just to have that first 5 minute conversation to see whether there is even any potential here.


To rewrite a comment I posted earlier:

There are always stories of senior candidates with years experience who fail painfully simple coding interviews. If these candidates were gainfully employed, maybe that indicates their roles, perhaps their entire careers, didn’t require programming? Or perhaps it means that it’s possible to write code without having a deep understanding? Or could interviews be simply that nerve-wracking that experienced people fail easily?

These widespread anecdotes should inspire reflection, not just on the utility of technical interviews, but on the very nature of the work that goes on in the software industry. What exactly is going on?


have my git in my CV, and the minute I hear I'm supposed to go through a technical vetting that includes coding I'm out.


Thank you. I always click on the GitHub links, and they are really extremely revealing. If you just send me your GitHub and it's good, then you're in the next round.


I've been reading these stories of interview questions for 10+ years now, and it seems to get progressively worse over time. It all reinforces for me the decision to be an independent consultant all my career. I get hired because somebody needs help thinking about their product/design, a project completed, or a difficult bug solved. We talk about what I propose for them to do, and I'll explain what I've done in the past that's relevant or useful. I'm not there to join the family, so I deal with Purchasing and Accounting, not HR.


I recently spent some time talking to other teams internally at a FAANG. I have 12+ years as an SDE and 4 in the company. One commonality I've found is that teams who do not ask coding interviews tend to become crappy over time. Even in an environment where you can look at individuals code history, product releases, and talk to their coworkers. I've taken it as a sign that the team values their own employees enough that they ensure that new hires won't be a drag, or have unreasonable expectations e.g. "I lead, I don't code" etc.


From the description, the test appears irritating for your taste but valid (if you pass you are able to reason about code in twisted, semi-realistic ways, and smart enough to win at multiple-choice exams).

Considering the bigger picture, this kind of coding test signals that the company really cares about you being able to code, to the point of pissing off some candidates in order to be sure about the rest. Moreover, it's a more cost-effective and objective test than a whiteboard interview: the company shows some competence at hiring.


There is a burgeoning culture of “automatic assessments” (and startups racing to develop and sell them to other startups) that mirrors the old aptitude tests thwt HR firms used to do when screening candidates for, say… accounting jobs (back before computers were a thing).

HR people love them because it saves them time and thins out the herd, some tech companies believe it sets up a minimum threshold to cull out inexperienced candidates, and, let’s be frank, many hiring managers are too lazy (or powerless) to either understand how these things work or fight the VP who’s been told “spend X and save X times 1000 in hiring the wrong people”, because who designs these things pairs them with a massively advantageous risk/savings model.

And most orgs will just spend the money on these tests instead of hiring people properly.

It’s like Hackerrank and their zillions of wannabes, who’ve spawned an entire subculture of cheats and people who can code just barely enough to solve canned problems, instead of actually understanding what the computer is doing.

(I politely back down from anything that even resembles something like this - like the old Google quiz interviews - because it tells me a lot about how the hiring process works and how much of a meat puppet HR would like you to be in that kind of company.)


If you get a chance, spend some time helping out with recruiting, interviewing, and hiring candidates some day. It will make much more sense from the hiring side of the table.

You may be fully qualified for this job and the test may be an unnecessary obstacle for you. If you only consider this job in the context of your own experience, requiring testing is going to feel like a dumb move.

However, not all candidates are like you. The hiring manager will receive a range of candidates from people far more experienced than yourself to people who aren’t able to write any functioning code without Googling for even basic syntax questions. Back when we did in-person coding interviews many years ago, it was shocking to see how many candidates would arrive charismatic and confident, only to fail because they couldn’t remember the syntax of a for loop in a language they (wrongly) claimed to be an expert in.

Look at it this way: If you work at a company that hired everyone based on their resume and nothing else, you would eventually be surrounded by a significant number of people who don’t know what they’re doing, or perhaps don’t care. They reach this point in their career by shifting work to people who do know how to do it and by draining the productivity of people around them. It’s terrible, and the good developers will leave.


I kind of enjoy the puzzles. One time I was having a slow day at work and a recruiter called, I took a tech interview in the afternoon just to have some fun with the puzzles lol.


They filter out positives, though according to hiring managers, that's the point. I had a frustrating experience with a hackerrank recently. My code was passing all of the public test cases, but failing one of the hidden test cases, and there was no way (that I could find) to debug it. Incredibly frustrating. Even if you're a long time senior developer who can code, don't be fooled into thinking you can pass one without practicing on the particular platform.


I'd be curious to see the data, but I think these tests are overall good predictors of SWE in the company.

Companies who use these types of tests (including FAANG) aren't stupid. Yes, these tests aren't perfect but no recruitment method is. There are many other constraints to consider and overall they are an acceptable solution.

I'd recommend to practice and take these interviews with an open mind. Otherwise, there are companies who use different type of interviews.


Convince the rest of the world to stop applying for senior+ positions and I'll happily stop giving coding exams.

Hiring is opening yourself up to complete strangers, literally anyone on the Internet can just submit an application saying whatever they want.

Even people with a decade or more of experience end up getting stuck on their specific tech stack and are utterly useless on every other stack.

How do you propose filtering through all of that, and why aren't people already doing it?


I hate timed coding exams.

The "Here's a thing to write, send it to us in a week, it should take a few hours to do" sort of thing seems reasonable. As an interviewer, I do a code review of that (or of GitHub code if there's enough available without an exercise), and then send the interviewee the EXACT QUESTIONS based on that review, that I'm going to ask, a day or more before the interview. I explicitly tell them that they're allowed and encouraged to research and can even bring prepared notes.

This process turns random questioning into, "OK, I saw some code, think through my questions about it and let's have an interesting discussion about it." This gives people time to look things up like you normally would when working on a project. It also seems to ease the candidate's down quite a bit, because some people are super great at their jobs and just don't interview well.


I think coding competency is really hard to assess in interview, and arguably the wrong goal in interview.

When I evaluated the real goal of interviews, it was surprisingly not to discover the best coders, rather to discover the best fit for the team. Consequently a chemistry/collaboration oriented interview was the most predictive of long-term candidate success.


We do a take home for every role. Managers do a presentation. Engineers do a coding challenge. We let engineers opt out of our challenge and offer up their own code. For the latter option, we review the code and do a live Q&A. We need a way to get candidates and just trusting a resume isn't enough.


Obviously answering the question: can this person that we're considering hiring to write code, actually write code? Is worthwhile trying to answer.

Whether you can reasonably do that in a 1-2h interview by asking them to type stuff into a computer, I'm not so sure. I suppose it's a not bad approach but it seems likely to produce false negatives. e.g. candidate may have been working in a non-coding role for the past year and might therefore perform poorly in a coding challenge, but when hired would get back up to speed in a short time.

I came from a hardware engineering background, and we had the same concern there: does the candidate know how to design hardware? We didn't ask them to design a circuit in the interview, but somehow it worked out ok to just ask them questions and listen to their answers.


I am not in web, but at least in my field, the "sort" question seems reasonable? You cannot always rely on the debugger being present and working -- maybe this is a remote service you cannot connect to, or maybe deployment takes hours while sending a test case only takes minutes.

As for programming questions in general, they are certainly annoying during interviews, but I would be hesitant to say they are net negative. We have hired a few people who just could not program, and this was very discouraging for the team: you spend month training them, telling yourself "I am sure they are smart, I just need to help them one more time" until you realize that no, you are just wasting your time. A few cases like those, and I can see why people want to ask coding questions on interviews.


Just putting it out there:

Are their companies out there that do a cultural fit/assessment (it's nonsense of course, they just want to make sure they like you) BEFORE proceeding to a technical interview? Why not save the candidate's AND interviewer's time?

Agreed that this might be a legal minefield, but for fucks sake, look at all the anguish the widely practiced interviewing practices creates.


I find them fun.

What's disgusting is when you solve the coding test with flying colors but the interviewer pulls the "candidate doesn't seem interested about X company" and still fails you. That should be illegal if the session you're in is called a Technical Screen.


Lots of timed algo ones in my city - i.e. time-space complexity. Completely unrealistic for a DevOps cum full-stack role.

If they were untimed - like homework assignments - I'd probably say yes to more but as it stands I've no issue securing work with top end rates after a f2f conversion (no technical vetting at all - people seem thankful to have me onboard).

I've seen reviewers reject 8-hour assignments because the submission uses a certain library or something contrary to the house style so I've stopped doing these too. Opinions matter but they shouldn't be a f2f filter.

Right now my hard and fast rule is no code on paper - sure we can whiteboard but I'm not writing as much as an if-statement unless I can hit F5 on my keyboard in my IDE.

(My second rule is no interviews in lobbies)


I haven't done any since I have not switched jobs in a long time. But I would not be insulted by them. I am not proud. I try to take an egoless approach to the work and try my best not to consider myself above anything. I would take the same approach to interviewing.


It used to bother me, but after getting an invite by a FAANG to do an interview I decided to go with it. Sure, I'm not a fan, but if you want to work at certain companies it's what you gotta do. So I swallowed my pride, took the test, and eventually got the job.

Worth it.


Candidates lie quite a bit and the rest of the interviewing nightmare trickles from there. I've had two candidates show up for interviews, who used the title Software Engineer on their resume, who couldn't write any code. That was a shocking discovery at the time and I'm someone who has worked in SRE alongside Operations folks who faked their way into SRE titles.

That said, the state of interviewing is completely unacceptable. The algorithm tests ensure that mostly college attending candidates pass them. Take home exercises are very burdensome for people with kids.

The company LightBulb that did a LaunchHN recently I think gives us the best hope for improving interviewing going forward.


I've been conducting interviews, we've experimented with adding some coding challenges and it's helped. Not live. But simple pass/fail write some methods to do a thing. This typically will filter out brand new folks.

But I do think you can have discussion, a technical interview back and forth - I'm talking details around guarding against nil classes, backwards compatible migrations, scale problems, real-world issues the interviewee has encountered. Types of tests they need. Maybe a situation where they missed a use-case.

If they are able to articulate a discussion like this, I'm pretty confident I don't need to run them through leetcode doing BFS/DFS binary tree junk.


Interviews are two way.

80% of people in the applicant pool are hopeless. And so are 80% of companies hiring for these spots. You should probably expect to turn down a number of companies you applied for. If you're at a senior level, the dynamic is reversed. There's probably more rejection by applicants than by companies.

Interviews filter for a certain kind of candidate. If an interview is all leetcode problems, then you end up with an engineering culture that only knows how to solve leetcode. If the process favors people who get shit done in 2 hours, then the engineers will mostly be people who only know how to hack things in 2 hours.


There seems to be a fair amount of hiring manager in this thread, so I wanted to ask a related question:

Is my current tech adjacent role preventing me from getting interviews as a developer?

I have been working in technology for over 15 years now. I am looking to switch but into software development and applying to roles that align very well with my past experiences and current ambitions. I've probably applied to 100 jobs and had 5 follow ups from companies! What could the issue be? My resume? Ageism? Current Role?

My current resume for context https://joe-riggs.com/resume/


I agree that understanding the problem space is exponentially more powerful than just being able to make code do what you want (or figure out why it doesn't).

However, I don't find this kind of thing insulting.

Unfortunately "how do you work with garbage code written by shitty programmers" is a pretty important question, though not one I have to worry about quite as much when I'm working on my own projects where I've written all the code. You might think that at a startup the garbage code wouldn't've had time to pile up, but that isn't my experience. Often founders have a better grasp of the problem space than of the solution space (I mean, they're often shitty programmers), and even their grasp of the problem space is weak at first; this can lead to multiple rapid pivots in which code that was good for one thing is patched as quickly as possible to be barely suitable for something else, with the expectation that it will probably be thrown away anyway, and if not, they can hire someone else to rewrite it.

I'm surprised by your thoughts on the bug test. Usually when I fix a bug, any bug, my first step is to construct a test case that reliably demonstrates the bug, which is what it sounds like they asked you to do. But you say, "this diagnostic technique of arbitrarily working backwards ... is probably the worst method of programming." What do you do instead? Or have I misunderstood your description?

My experience with hiring programmers (18 years ago) was that a short emailed test with a few simple programming problems I could do in a few minutes was enough to easily rule out about 90% of applicants, including people whose credentials looked great on paper. Of course it tells you almost nothing about their understanding of the problem domain or their ability to design code to be maintainable in the large, but if they can't program then they can't do the job even if they're good at those things.

One deficiency in our process at the time was that we didn't test for the ability to read code, just write it, and we generally spend about five times as much of our time reading code as writing it. The debugging question sounds like a useful technique for giving some insight into that.


I’ve been a hiring manager at both startups and FAANGs and code tests are absolutely useful. The only candidates who I have ever heard complain about them were ones that couldn’t pass and blamed others for it. Things like forgotten return statements in C++ causing stack corruption were blamed on the boilerplate test code, and every time we dodged a bullet by discovering bad candidates.

I honestly don’t get how anyone could be offended by a coding test. It’s what we do as programmers and I’m deeply suspicious of those who refuse.


I’d prefer to earn a badge from a single coding test on leetcode or similar. Then companies can have more confidence in applicants’ skills and skip the code tests.


I find it insulting but that's the way it is.

The problem is it is very hard to fire a bad hire, sort of joining a fraternity or sorority. So they add rituals before you join them. If you can take the abuse for joining them, then you should be able to conform to their way of life/work.

Unfortunately, you have to play the game or start your own company.


If you dont know that lots of people lie in their CVs then you are not a senior engineer. (Also how come a senior engineer never did any recruitment to learn this?).

Also you must be very lucky to not know that often the senior engineers are a problem - they got promoted duebto seniority, not any actual skill.


I interviewed for a 6 month contract and there was a programming exercise. My first question was, what unit test framework is available for the Python code they wanted me to write? “Errr, there isn’t one”. Ok, let’s create a simple one first then. I was offered the role, but I declined it.


Oof. The online test. Probably the auto-graded kind. The kind that pushes all the inefficiency of the process out on to the developer.

Well, take it if you want. I'll never do it. But then again, I have missed out on what might have been great opportunities because of that.


Whether the tests are useful or not, if a prospective employee - no matter how senior - felt insulted or thought it was beneath them to answer simple questions, I don't really want them working for me in any case.


one time some late-teenagers wanted to focus on FizzBuzz instead of the portfolio of code i wanted to walk through in full discussion mode. from a peer who already worked there, i should be glad i did not get an offer.


Nope. I don’t see things as “insulting”, in general. It’s just cost-benefit. If it’s a great business with a great team, I don’t also need them to be great interviewers. I’ll just fix that when I join them.


Frankly, I wish there were a way for me to give a coding test to devs at companies when I'm applying.

The gates to company entry are huge and solid and golden.

The court inside the gates is often unkempt and dirty and strangely vacant.


Only the one I took 2 years ago that had me implementing a marquee tag


If I ever look for a job again my plan is to give the hiring managers my performance reviews from my previous company. There's enough information in them that speaks to the kind of engineer I am in the real workplace working on real projects with real people. I think that + a 1-2 hour conversation to verify that I know what I'm talking about (i.e. I didn't forge the perf review) should be sufficient for hiring manager to assess if I'd be a good fit in their role. I think that will be a lot better/easier for everyone.


The problem with this is that your previous manager may be an idiot who can't accurately judge your skills. You have the same problem: your manager is no more trustworthy than you are, and it's much easier to just have you write some code in front of me to solve this problem.


If you don't like it, that's fine, but they clearly are not trying to insult you. You're taking it personally. Just because you're the best developer in the world doesn't mean you need to take to HN to rail about their crappy hiring process. Instead try sending them a nice email explaining your concerns. Help them craft a better hiring process. Or find out from them why they made it that way.


Hiring manager of hiring managers here. This thread is full of managers pointing out it's hard to work without the tests and IC's who don't have much empathy for the constraints managers are under.

Basically, getting a job for a senior developer is the easiest thing ever to require so much work. You're in hot market demand, but if you spread your search out to include multiple companies, the hours of screenings, interviews, paperwork, and coding challenges start to really add up. Clearly that's frustrating.

For managers, hiring a bad fit is simply disastrous. Turning around and firing them is never easy. Yeah, it's slow, and that person drags your team while it happens. If you can make it fast, that has its own affect on your team culture. I understand this is an unsympathetic observation, but: firing people is emotionally hard on managers. So, yeah, we'd rather pick right the first time than play a "fire fast" iterated numbers game.

I'd propose that the existence of the test isn't the problem, it's all the stuff around it. As a hiring manager, my job is to ask you to do this thing you don't want to do, and to explain why it's important for the company and the team. This is the first time, but not the last time we'll have a conversation like that. You should note whether or not I'm good at it and whether you'll tolerate that kind of thing from me. Because as an employed engineer you'll end up tolerating those requests from someone.

As a candidate, you can stand out by making a deal. "Hey, I'm really excited about this opportunity. But I have several companies in my process plus I'm still employed at my current job. I understand your side—a resume doesn't always tell you if someone can code, and I definitely don't want to work with other people who can't pass something like this. Still, I need to manage my time well, and this is too much. Is there any way we could cut the size of the challenge in half? Or, I've got an open bug on an open source project I contribute to. Can I work on that, put in a PR, then discuss that in the interview? Trying to get us both what we need here. Thanks."

Personally, I'd respond really well to a discussion like that, and I think all my hiring mangers would. And the discussion itself would make me want to hire you more: you're a team player, you're transparent, you aren't dismissive of me but you'll advocate for yourself.


yeah. coding is ubiquitous, and tests show it. dev is an art and a science, and tests can easily screen those out. but that's why dev is rare and valuable. and that's why dev will eventually find its place, without tests. meh.


Yes. I refuse to take them.


I've accepted that real world development and interviews are 2 different worlds with little overlap.

Like many things in life, best not to think too hard about how things should be and focus on making the best with how things actually are.


Yes.

An algorithm is not a valid test of your skills and experience. e


In my staffing/hiring experience (about 40 people) I find its very easy within about 10 verbal questions on a standard voice call to determine some one CANT code. How well can they code? That takes more than 10 questions but is equally easy to determine.

Keep in mind this is only useful on mid to senior level people with a lot of experience. You'd never interview a fresh bootcamper or CS grad this way.

For example - Im hiring for a position where I know the job and the tech requirements because I've done the job myself. Im only phone screening candidates who's resumes paint a picture of someone who works in this area, in these languages, in this tool chain etc. I'm not screening randoms off the streets here - Im actually bothering to qualify people before I bother to screen them.

Lets say its full stack web development and I expect the candidate to be a producing team member in 30 days (because if nothing else, nobody off the street knows about our business domain, what the hell a TR-111 device is or a domain specific message format etc - not even to mention where all the bodies are buried in our codebase). Theres a certain baseline set of things a full stack web developer has done more times than they can remember. These questions are intentionally vague.

1) Whats the difference in using the standard GET/POST Request/Response style of web page vs a page that does the same thing via invoking a backend API? Tell me how you build one vs. the other?

2) Someone asks you to build a screen that fetches some data from the database - walk me through the various considerations and how you do this - Give me step 1, step 2, step 3 etc? What if there are search parameters the user can enter? What if this stuff is sensitive and I don't want it showing up in any log files? Obviously theres more than one way to do this so any real answer is a good one. This question covers a lot of ground and almost does the whole job for you.

3) How do I debug Javascript in a running page?

4) What are the most common Bash commands you find yourself using? Any automation scripts you've written?

5) Why cant I rely on developing with some whacky font on my local machine?

6) Talk to me about CSS media queries. What are they for? When do you use them?

7) Language specific stuff like when to use a Hashtable vs. a List? Whats a linked list?

8) Say I give you a hostname and some creds to a remote machine and I tell you theres apparently something going wrong on the prod web app. Where are you going to look, what are you going to look for and how are you going to look? ("I'm going to ssh to the machine like user@hostname ... then dig around /var/log ... ") - bonus if you can tell me where to look next and the likely problem as a result of this search.

9) A junior dev is having trouble getting a server running - some error about a port conflict. How can you help?

10) You get a ticket that requires you implement code requiring a new dependency - tell me how you get all that running locally. Does this have any deployment implications?

If the candidate gets this far you can be pretty sure they really have done the work specified on their resume. Now you know they can code. One does not come by knowing answers to questions like this without having written some code.

Now - how well can they code? This takes 1 hour during an onsite interview.

I recall one interview where this was done very well - the Product Owner handed me a stack of printer paper and some pencils.

"I want you to write as close to full syntax as you can without an editor, an end-to-end login page where I get a login failure or a page that shows my address from a database. Think of every page as a file you'd write for your solution. This shouldn't take more than an hour."

It didn't take more than an hour either. I had a job offer by the end of the week and I joined a staff of people who had done the same thing. Turns out we could all code just fine.

Now, this is an easy kind of interview if you're talking to a person with your desired qualifications. If you're not then its a very short interview. From the candidate perspective, the ones that represent themselves accurately in their resumes knock it out without issue while having spent the bare minimum of time with the process. No take home assignment. No obtuse CS topic that a senior dev hasn't thought about in literally a decade. No contrived problem with a bar, 2 billiard balls and a fulcrum, etc. Just simple, fast and to the point.


Coding interviews are why I always consider leaving this industry despite the pay.

I've spent 10 years accumulating a large body of knowledge regarding product and system design, various tools to do it, and a mastery of a few languages.

Yet, instead of being asked things relevant to my abilities I'm asking coding questions that are designed for junior fresh grads. No, I don't remember all of my algorithms. No, I don't know the newest technology google has released. Do you want me to build you something? Great.

I'm in my mid-30s and I cannot see myself learning every new tool some edgy start up or FAANG vomits out unless it's so abundantly useful I cannot help myself. I don't want to spend weeks grinding leetcode. The last time I interviewed was 5 years ago and while I have gathered a vast body of knowledge on engineering and am extremely capable none of this would ever show in a leetcode interview.

... and don't get me started on the 8 tier 1 month long crucible interviews where I'm subject to several coding interviews, an architecture interview, a personality interview, and several other meta-meta interviews. Oh, and this is of course on the company's schedule so it's always inconvenient, sometimes requires travel, and is always an incredible pain.

I'm convinced 99.9% of hiring managers are failed engineers that failed upward into management. They themselves could not pass the tests they give, so it's being used a cudgel to have one over on the poor sap who needs to pay rent. I mean this to be insulting. I have very, very little respect for engineering management vs ICs. IC roles are earned through actual talent, engineering management is political.

The entire industry is built this way and it's insulting. I can understand thoroughly interviewing someone with a questionable pedigree but a simple call to any of the half dozen references I have in my resume will confirm my ability. Being treated like a liar and a salary thief and given the 3rd degree is an insult, absolutely.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: