A three-hour test is above what I'd consider acceptable for most potential employers. I can talk with you a bit, but if you want me to come in and do work for half a day, I'm going to refuse, I've got actual paid work to do.
I never looked at the memcached source, and I'm not even an especially experienced C programmer, but I'm pretty sure I can do it in well under 3 hours.
1. grep -r '"append"' to find where it does "switch (cmd) { .. }" – append seems reasonably unique, but maybe there's a better command.
2. Modify switch to recognize "mult" command.
3. Find "append" function that does the work; copy/paste, modify to do multiplication.
4. Converting two strings to a number, multiplying the result, and returning the result as a string is a fairly trivial function.
Unless the memcached source code is hugely surprising, I might be able to get a first working version in 15 minutes or so on a good day if I don't mess up, maybe double on an average day. Add 15 more minutes to build memcached and read the question, and maybe a further 15 minutes of making it nicer/leeway.
A 3 hour limit is long enough so that no one reasonable will stress about running out of time, affecting the performance, and short enough that bozos can't easily trial/error it, ask on Stack Overflow, or whatnot.
Other than that, the stage matters a lot as well – is this the first interview question shotgunned to everyone who applies, the final one and you're hired if you pass, or something in between?
You also need to add tests as well as decide whether to break the existing API, or to create a new function in the API, in which case you need to design it. Do you add room for future support for division? Exponentiation? Knuth's up-arrow notation? Logarithms? What about integer factorization and discrete log? What about factorial? Sooo many choices. I can easily waste 3 hours in just 1 hour.
None of that is in the description. You are spectacularly over-engeering things.
A test might be useful, but I took a quick peek at the source and it really is as straight-forward as I suspected it would be: the input is already tokenized and you can just add a new entry in a large, if .. else if tree, write a multiplication function, and that's pretty much it. You can just test it by started memcached and sending it commands.
I'd whip up a quick patch, but the tar.gz is 14 years old and doesn't compile with latest clang without twiddling -W flags that don't seem to apply, and the git copy doesn't compile due to autoconf woes (sigh...) Messing around with autoconf is not my idea of fun and too much pain for a HN comment, so whatever. But you really don't need 3 hours if you vaguely know what you're doing and are a competent C programmer (which I am only barely).
Right so – that wasn't clear to me. You'd be surprised how much some people will over-engineer some interview questions like that (even when you very explicitly tell them to use the simplest thing possible).
The memcached protocol really is as simple as "CMD [params]", all of which is parsed for you, and this API is just "MULT num1 num2". There are no tricky bits or hidden complexities here as far as I can see.
A little hint: multiplication and addition are associative which is why they make sense as atomic operations. (Although allowing both multiplication and addition in the protocol breaks this ...)
If you read the introduction and then spend extra hours doing "other things" (that are bad), it could count against you.
Don't forget to add a test (written in Perl). It's not required by the actual task description but it might give you an edge over the other applicants.
That's why employers need to spend more time convincing potential candidates that the role they hire for is actually worth the trouble of going through this kind of coding interview test.
In the past I've encountered companies that would just throw a multi-hour coding test at you without even giving you any idea of the role they are hiring for. If it's not worth employer's time to get the candidate interested in a position, the position is not worth applying for.
Not to mention that some employers are moving this testing (whether you agree with it in general or not) to the front.
Used to be that it might be "Hey, let's have some conversations and then there's a little project..."
Now I've had multiple employers open with "Thanks for applying. As a first step, we would like you to perform this project. From there, we will let you know of next steps."
If that’s what you’re selecting for - fine. You’d be missing out on people who need to care for kids or other relatives and on people who assume that a company that requires you to invest significant unpaid time on top of the regular interviews is just not worth it.
Even “experienced job seekers” might have the time to do 1 or 2 of these “tests”, but typically you’d apply to more than 1 or 2 companies while looking for a job.
In practice, I am both too busy to waste a day on a programming task just to secure a seat at an interview, as well as willing to spend days on a cool problem. It all depends on how much I want the job and how much the actual spent time is my choice and not expected.
Part of my argument is that not everyone has the choice to spend an additional 3 hours per interview, meaning a process including tasks like these discards these applicants.
I think if the company is posing an interview question that requires total fluency over a legacy version of C++, they're probably not looking for graduates.
C++14 is a safe bet. C++17 is almost pushing it. C++23 isn't even fully implemented by any compiler.
I tend to default to C++11 with C++[11,23]-portability.
In general, write maximally portable code without using vendor- or standard-specific features without a/some very good reason(s).
Also, C++11 is a monumental improvement over C++ from the days of rusty sharp edges of 90's C++ where one needed to follow strict standards to avoid nonobvious UB.
You sound really full of yourself and entitled. A job should last several years, why not spend half a day applying? You don't sound like you really want a new job anyway, but maybe try to put yourself in the shoes of people who do, or who actually _need_ a job.
Because what are the odds that I'm going to land this job? Surely not 100%. If they were, sure, I could spend half a day applying.
But what if my odds are only 10%? Then, on average, I'm going to have to apply to 10 jobs to land one. 10 x half a day is a full working week. Make that two weeks if my odds are only 5%, three weeks if they're only 3%.
That's a problem, especially if I currently have a job. I also have a life; I don't have three spare fulltime weeks to burn to try to land a new job.
I’d say if it takes you three hours you’ve already failed.
I’ve fixed a bug in memcached (improperly cleared buffer, null GETs resulting in zero prepend on next GET under high stress) during a five alarm fire and deployed it across a cluster in less than 30 minutes - and I would consider myself a mediocre devops hire.
How would I know that the task actually takes 30 minutes before scheduling as much time they say the process would take? I presume that when they say "come in for an hour of talk and a three-hour practical task" (which the point where I'll refuse) I won't even know the task and can't already assume that I can actually be done much faster.
Right, sorry, I wandered off into the woods while writing that. They should allocate 30 minutes. Even if you don’t finish, by 30 minutes there should be a clear indication of your approach.
I've never had such a panel interview in my quite long career and possibly never will. I'm not allocating a whole day for an interview, that job offer isn't likely to be that much better than the current job or other alternatives to justify jumping through such hoops.
If you want me to do work for you and judge me on that, I'm available as a contractor.
You have never had an all day interview? Not even onsite?
That’s weird. Seriously. That’s weird. Every job I have had in my 20+ year engineering career had a multihour interview panel. I’m talking like at least four separate hour long interviews all on the same day. Big company. Small company. It literally didn’t matter.
And you’re saying this has never happened to you? I have to ask, what has been your typical interview experience been, and how tiny are these places?
I did just literally say that this has never ever happened to me. Likely that's a regional thing, perhaps panel interviews are popular in USA - but my practice has always been 1-3 rounds of <1 hour interviews, at various sized places including a multinational bank (where I also did many interviews from the hiring side both for technical and nontechnical roles, but it always was ~3 rounds of <1hour interviews with no take-home tasks; if more people were needed, they were included at one of the interview rounds, not as extra time), a major government institution, and a matured tech ex-startup - and while your question was reasonable, IMHO framing your prejudicial assumptions as "how tiny are these places" was an unprovoked insult that's not entirely appropriate.
No. You earned the “tiny” comment. You started with being super arrogant, on a site known for arrogance, known for people flexing about how they get their jobs from their friends, on a site that you know is not only being very US centric, but very Dan Francisco centric, speaking on a very common topic here. Legitimately, the only reason beyond nepotism I could come up with for your very strange experience was you never worked anywhere larger than 3 people. Again, something that is quite common on this site.
At some point, you have to wonder when know you’re the odd one out, but you didn’t. That’s on you.
Frankly, I think the only place such interview panels or loops really make sense is in something like Product Management, where you might need to interface well with Sales, Marketing, Engineering, UX, executives.
Agreed - this is a great type of question. Being able to dive into a new codebase and read code at speed is a critical skill. Even when working on "the same old codebase", sometimes you run into weird corner cases that have you diving through a dependency, or debugging a perf issue that has you walking through other parts of the system. And reading a codebase doesn't always mean grokking the whole thing - if your goal is to figure out why some API call is failing, targeted reading is exactly what you want to be doing.
FWIW: Took me 20 minutes from start to finish, without editing tests, but including ~5 minutes taken to update the codebase so my new-ish Clang would actually compile it. I didn't touch the binary protocol, though; in a real setting you'd have to make sure the binary protocol extension would be compatible with other extensions made since this version.
No. But I don't often work on databases so I don't find myself grasping for any commands in a cache. It's hard to imagine that if there's a use case for adding a delta there's not also a use case for multiply by constant. But I'm not familiar with the market so maybe I'm wrong.
The one I can think of is tuning circuit breakers so that failures are additive but successes decrease more multiplicatively/exponentially, which helps with fast recovery.
Cool challenge, but isn't it a bit of a red flag? I don't expect you to ask me to get familiar with a codebase, test, and implement a feature in 3 hours. Ever. It's also oddly specific, if you know what I mean...
> I don't expect you to ask me to get familiar with a codebase, test, and implement a feature in 3 hours.
In any given day, I might find an issue or missing feature in one of my dependencies, clone that repository, make a change, submit a pull request, and move on with my day a few minutes later. This is absolutely an important skill.
I don't expect someone to be able to become an expert with a codebase in 3 hours, or even become deeply familiar with it. I do expect someone to be able to get familiar enough to make a simple change, and to feel comfortable working in a codebase that's new to them.
If that thing is in production I wouldn't just change the code, submit a PR and move on with my day. For interviewing purposes it could work, but in the real world it's a reckless attitude.
What's your alternative plan for when a dependency is discovered to be broken?
I've done Josh's plan a bunch of times. It's why I prefer my dependencies to be open source. Everything is broken, but if it's open source, it's easy to fix it when it's noticed, push to production to fix it for my users, and send a PR to fix it for the world (eventually, if they take my patch; not everyone does, but I've had many accepted)
Nothing reckless about submitting a PR - that's what code review is for, to let people discuss changes with other people to reduce the risk of bad code being committed because the developer misses something that someone else could have spotted.
That's part (not the only) of the signal thing being evaluated: can you efficiently familiarize yourself with a codebase. I'd encourage you to try it. It's not that big a codebase: 140 files, 80k lines of code. Given that similar commands (incr/decr) exist, finding a way to get multiplication to work shouldn't be out of most qualified candidates' skillsets for a software engineer job (language skills aside).
My biggest criticism of the task is that the article doesn't make the goals of the task - i.e. what the signals are that they're looking for - clear at the start. You can see in the comments that a lot of people have gone in looking for a kind of trick and made the whole thing more complicated than it needs to be.
I can imagine in the real interview, there'd be more build up to the task, and a description of what the interviewer was actually looking for. That way the interviewee understands what the purpose of the exercise is and doesn't end up lost down rabbit holes.
The more I think about this idea, the more it grows on me though, if you frame it well and find a task that can be explored in the given time frame.
Note, I haven't used C in 15 years, have never looked at memcached code and in fact the last time I used memcached was before redis even existed.
It took me ~26 mins which included reading the article, googling the memcached repo, downloading all dependencies, compiling a first build, grepping for references to incr and adding mult with support for negative arguments, refactoring the guts of incr into two functions to support the changes, compiling, fixing a few compiler bugs then testing, given a few more minutes I'd probably be able to get it PR ready with a full test suite.
I presume that the poster did not get the job. The correct answer is that while addition and subtraction are mutually order independent addition and multiplication are not. This cannot be implemented for a value that allows addition. You need to create a new type of value "multiplyable" and implement multiplication only for that new special type of int. Otherwise you'll end up with meaningless race conditions in the case,of multiple clients.
Congratulations, you have qualified as a Type 1 candidate and did not get the job.
"and the whole thing needs to be atomic. Let’s look at how the locking works…” They spend all three hours getting deeper and deeper down various rabbit holes, and never produce anything that works. Candidates in this group don’t get hired."
The article mentions an `append` operation which is obviously not commutative; does that imply memcached needs a separate "appendable" string type? Of course not. Memcached only guarantees that the individual operations are atomic, it's the client's responsibility to avoid race conditions between multiple operations.
Now to in order to multiply a sequence of numbers correctly in a "high performance" shared store, you have external process synchronization. Just awesome. Thank you for the lesson. I had no idea concurrent processing could be so easy.
As another data point, I’ve faced this question as well when interviewing with them.
Having said that, it’s a great alternative compared to Leetcode style questions, where you’re penalized for just asking about the problem or not having the right insight from the get go, or not using the interviewer’s preferred formulation of the solution (such as using two queues for a graph search problem instead of just one).
Maybe it's just because I do that a lot, but imo potential candidates that are HN readers already know how to do this, if there are no gotchas. I find leetcoee considerably harder.
Yeah I would be fun to do a analysis of commenters, my guess would be that people defending leetcode style interview are probably younger and people praising a real code change are older simply because that's what each group is more used to.
Is there any other field that normalizes this type of insane interviews.
They are even applauded by the vast majority in this thread.
Do mathematiticans have to interiview like this? Medical doctors? Police officers?
Rocket scientists? Chemists?
Architects?
I cant think off a single profession apart from the IT domain that does this.
I dont think there is a single profession that gets mocked on their interviewing strategies as much as IT still I see senior devs defending this insanity.
'We' as an industry also seem to be very allergic to any sort of formal certification process, so this is where we end up. On the positive side that means that in theory anyone can show up with no formal training, pass the interview, and get a job (as I'm sure many of us here have). That is literally impossible for all the other jobs you mentioned.
We either have to start requiring the same level of formal certification we require of other engineering professions or we have to deal with these types of interviews.
And let's not pretend there aren't plenty of bozos who don't really know what they're doing in other fields. It's entirely unclear to me whether certification helps.
It's entirely unclear to me whether certification helps.
I've worked a lot in civil engineering, and honestly the worst working civil engineer I've worked with was still a lot better at their job than the worst working programmer I've worked with.
> I cant think off a single profession apart from the IT domain that does this
The dichotomy lies in the fact IT and programming are not professions.
All of the professions you listed, you have to go to school for. For most of them, there is a body overseeing training, qualifications and conduct of the professionals, barring them from practicing for malpractice.
How do you evaluate Bob, a self-taught, remote developer, with no public projects, with 10 years of experience in no-name companies? You give them a real-world task and see how they do.
There's a ton of nuance to it. It's all about context and with what confidence can you derive that a candidate is suitable based on the available context.
Alice might have the qualifications you listed, but never worked in fintech startups and therefore needs additional probing. Bob on the other hand might have none of those qualifications, but has worked exclusively in fintech startups, and would therefore require different evaluation.
This kind of context-appropriate evaluation ofcourse becomes prohibitively expensive when you're filling hundreds of positions per month.
The issue is without it how do weed out the bullshitters and frauds?
I've done loads of interviews of "engineers" who could barely program at all. A short test like this can remove them completely and then you just need one discussion interview for the handful of candidates that remain.
I do agree that this probably has no value for the company. They could implement this easily themselves. Now, if they would give a different arithmetic operation to each applicant...
I think the question is, why don't other fields put this huge emphasis on the fear of bullshitters and frauds?
My take: There's some extreme sampling bias at play. Everyone knows at least one horror story where someone managed to bullshit their way to a position. Now they imagine that this must be a problem, while ignoring the vast majority of devs they've worked with that were legitimate developers.
The field of software engineering is also much more personal and opinionated than traditional engineering. This lowers the threshold for what devs. will classify as "frauds" - not because the people they accuse might actually be frauds, but because they are not up to their standards.
If your goal is to remove the bullshitters wouldn't a 5-10 minute, slightly harder than FizzBuzz, test also remove them just accurately and save everybody a bunch of time.
It's really not at all unusual: Engineering roles in general will often have some technical interview with a problem to solve. If you are interviewing for a role with a particular practical skill (welding, soldering, plastering, etc) involved, it's also not at all unexpected that you will demonstrate it before you are hired. There's this idea that some programmers have that programming is some kind of aberration as a field: there's issues or practices in software you find nowhere else. And generally, this seems to come from a romanticised view of other fields: of the fields I am familiar with, software is not a particular outlier in interviewing, estimation accuracy, or general smoothness of project management.
The difference is that traditional engineering roles will not ask you to solve partial differential equations or show the proof of some equation on the blackboard, but rather ask fundamental "big picture" questions related to your work experience, or the field itself.
Not sure about police officers, but the other professions you named have to complete graduate or post-graduate studies in their field, pass a formal examination, maybe publish a few peer-reviewed papers while architects build a portfolio - this is usually proof enough of their strengths.
Actually, this works in IT as well: if Donald Knuth, Yann LeCun or Zvi Gallil ever decide (however unlikely) to apply to a web shop, they'll probably get waved through on the strength of their credentials. For the self-taught J. Random Hacker, who can't show any of his old code because it's under NDA, there's the interview pipeline.
I have worked with people (well, consultants) hired based on the fact they breathe - I would much rather jump thorough some hoops than to have to work with them ever again.
This could be why programming is seen as arts. Artists have to show a portfolio or a reel, that takes hundreds of hours to create. And then great style, a good story and crisp explanations on top of that.
The awful thing though is that you’re not allowed to show your old employers’ code. And most of it is teamwork.
I can tell you, that coming from a more traditional engineering background (electrical / control), I never - ever - encountered these types of questions.
Even though the real-life work tasks could have catastrophic or extremely expensive consequences. As in shut down production on an oil rig.
I've also worked as an analyst / data scientist, and didn't see too many leetcode-like questions there either.
I think one big reason for this is that other professions have some quality assurance of their candidates through their education or certification. Whereas many companies are open to hire self-taught and similar people...yeah, I'm not 100% convinced about that either, because somehow companies will assume that a CS grad candidate can't code for shit, whereas traditional engineering companies will assume that a engineering grad candidate knows something.
Welders usually do a demonstration weld as part of the interview.
Architects bring a portfolio, and walk through it. And licensed architects have a majorly huge ordeal of a licensing process. After obtaining an accredited degree, you have to get signoff on performing 3,740 work hours in six areas of architecture practice (AXP, architectural experience program). You may need to work at multiple firms to get this experience, since not all firms do all types of architecture work.
You'll also need to take the ARE (Architect Registration Exam) and any state suplemental exams. The ARE is actually 6 separate exams, the shortest is 2 hours 40 minutes; total test time is almost 20 hours, but with breaks you're looking at 24 hours 20 minuted total appointment time. Most candiates prep and take one test at a time, but you've got a 5 year rolling window to pass them all, and the tests change from time to time. I'm not looking up how long the California suplimental exam takes, but I'm guessing it's at least a couple hours.
Some of the test is multiple choice stuff, but there's interview based tests and probably some practical work.
If you're thinking of becoming an architect because interviewing will be more straight forward, let me save you some of your life and say, don't do it. Anyway, most of the fun work is delegated to juniors/interns; if you want to make reasonable money, you've got to get up to partner, and then you're only doing client relations and very little design.
I took a government provided standardized test of computer knowledge to work at a school district in 1997, and it was full of useless questions about early 1980s tech. I can only imagine the depths of forgotten knowledge that would be needed for a Software Developer certification program. How many hours of debugging BASIC or bringing up a 68000 board from scratch would we all have to do in our work experience program? Or formal methods: in my 20+ year career, I've never gotten anything remotely close to a specification that could be used for formal methods, but I bet I'd need to get work hours and study up for the test.
Oh, so another interview question where a Eureka[1] moment is required. And, don't tell me: You got it right. Next, you are write about it how it is such a "genius" interview question to select the 10x engineer. This question looks ridiculously hard, considering the setup.
> Type 2 looks at the problem and says, “Ah! I know just how to do this! Multiplication is just like addition, except wherever addition does +, I should do *.” So they copy-and-paste, change all the +s to *s, and they’re done in 90 minutes. Candidates in this group stand a really good chance of being hired.
And just before that, he wrote:
> By the way, MemSQL-in-2013 was doing deeply arcane and high-performance C++11, so the fact that this challenge incidentally requires fluency in C was a plus, not a minus, for their purposes.
I don't know a lick of C/C++ and I found the add delta method in <10min after poking around the memcached GitHub repo. I feel confident I could complete this challenge in 3hrs, or at the very least make a strong attempt.
I think it's a good question because it heavily indexes on strong engineering fundamentals. Searching a codebase, reading code, understanding behaviour, modifying an existing codebase, blending your solution in, etc. Every strong engineer I know is very good at the things required to complete this task.
I don't believe this falls into Eureka moment category. This is adding an operation into a codebase which already implements a similar operation.
The problem is also not "fluency in C" because they require it for day to day work.
If anything sucks about this task it's that it also tests "set up a C IDE from scratch on Linux". This might reject a good candidate who nevertheless didn't recently set up such an environment. OTOH this one is easy to fix - either set something yourself or tell the candidate they will be expected to do this, so they will have time to prepare.
There's no Eureka moment required. "Add a feature similar to an existing feature to a codebase you don't know yet" is what people often do during their usual day. It's a good test of quick exploration, being ok with uncertainty, and coding. If that's what the company is after - great.
Yes, it's a hard question, but if you're grepping for skilled programmers you want a hard question. The point of these types of tests is not to minimise false negatives, but eliminate false positives.
(And besides, "I need to replicate this addition-based functionality but for multiplication" does not seem like such an Eureka moment to me. The main difficulty for me seems to lie in navigating a big codebase, and exposing a new function on the API -- the difficulty of which depends on how well modularised the codebase is.)
> Via its incr and decr commands, memcached provides a built-in way to atomically add to a number. But it doesn’t provide other arithmetic operations; in particular, there is no “atomic multiply by ” operation.
Going from that to "maybe I should implement multiply similar to incr" is not a big enough leap to call it "Eureka" in my mind.
there were three possible groups given, the two groups that solved the problem both had "eureka" moments. Group 1 eureka was so bad I would definitely never want to hire them. Group 2 eureka should not really be an eureka moment for anyone over 12 years as it essentially hinges on "Multiplication is just like addition"
Although if it was me I might be removed from consideration because I would probably just send an email - what? Am I missing something here? It sounds like you want me to just copy the add function to new function mult and change the addition sign to a multiplication sign? Is that it?!?
Three hours? My own time was about 45 minutes (at MemSQL, did not get an offer), completely unprepared. It's not a complicated codebase to comprehend, and you can rely on the existing plumbing for other ops. The best candidate I've interviewed also implemented DTrace instrumentation support for this in the same amount of time. I do agree this is a great question, far better than irrelevant Leetcode bullshit that gets typically asked. At least it's far more predictive of the actual on-the-job performance, because this is what the candidate will be doing mostly, as their job.
I'm not a TDD (test driven development) zealot but this kind of problem is begging for a test harness. Even if you don't want to dive in and figure out the memcached test system you could write a script that did something like this to greatly speed up your development:
(echo "set age 0 3600 2"; echo "mult age 10"; echo "quit") | telnet 127.0.0.1 11211 | diff expected.txt -
and then create expected.txt to test the output. (Note: I created the command line from memory - maybe nc would be better to capture stdout.)
Telnet reads keypresses from terminal instead of using stdin/out. You need to use nc instead. Or expect, but that's stacking one terrible thing on top of another...
> Basically, this question is to software engineers as FizzBuzz is to programmers
I don't get this comment at the end or the follow-up article? What's the difference, sure, FizzBuzz is a simpler problem, but I would imagine it being a common interview question for software engineers
The point he's making is that software engineering skills are about a lot more than just programming. Being able to navigate an existing codebase, implement new functionality and blend it into the surroundings is an important skillset that is pretty distinct to just writing code.
That to me still feels very much like a programming task.
To me, broadening to software engineering means things like requirements gathering, timeline estimation, prioritization, etc. If you just wanted to focus on code I'd choose things like writing programs that are extensible over writing extensions to existing programs.
And yet many people on this post are claiming that they cannot tackle this task for a variety of reasons non-programmming reasons (I.e. they don't even get to the programming bit of software engineering).
Sure, it might not hit everything but it hits quite a lot of important skills.
Companies that plan on doing something like this: please make sure you are paying your candidates market wage for the interview.
For Leetcode style questions, there is some amount of interaction from the interviewer, so the cost to the company is that employee's 1 hour wage. If as a company, you feel like this is an efficient and superior way of interviewing, consider giving that amount to the candidate.
This question is asked over an interview (I’ve personally faced this question) and the company isn’t deriving any more value out of asking this question as opposed to a Leetcode-style question, so I disagree.
For OP he said it was three hours after the context was given. I guess it depends, this is personal to me but I'm bowing out of most processes that I deem to be meaningless/a waste of everyones time.
Then of course, the natural conclusion is that some portion of potential candidates would always be dissatisfied with the interview process and question set, which maybe explains the persistence of Leetcode-style interviews with all that entails.
For a “first level” interview the time demand is probably unreasonable.
But if you’re already shortlisted, I don’t feel it’s an unreasonable amount of time for a relatively basic assessment like this before they commit to paying you a significant amount of money by employing you.
That implies that their assessment of their hiring process is not in fact incorrect, broken.
Companies (as in its employees esp managers, processes, culture) do things that are not at all efficient, correct, in the best interest of the company all the time. Not to break out into another discussion - but the resent push to force employees back to the office could be seen as one such example.
> If companies thought they were missing out on too many good people they wouldn't do it.
that doesn't reflect my experience at all.
from the outside many weird things seem to have some deeper meaning and thought-out plan. But when you get an inside-perspective, you notice that often no meaning or plan exists, and the reason for the weird thing is something silly like "an important person made a suggestion, that someone else misunderstood, but also made it a priority", or "there was a good suggestion, but it wasn't implemented, because the person who suggested it, was in bad standing with the management".
Or maybe candidates should submit their answer as a PR to memcached. The interviewer is deriving some value from an open source project, they should contribute to it.
I feel like that could very quickly devolve into a Hacktoberfest spam situation where the memcached maintainers keep having to close unsolicited "add a multiply command" PRs. The feature used in the question was designed to be easy to understand and implement, not actually useful to anyone.
> make sure you are paying your candidates market wage for the interview
I always wondered how workable that could be at any scale. We sometimes hear about company straight contracting small fixes to a candidate and use that experience to decide on hiring.
But most job interviews I've ever received where while being in a full time exclusive contract with my actual employer, and I wouldn't quit before getting the next job. Doing some coding interview on the side is of course fine, but the moment money gets into the mix, I feel it becomes a weird line to walk on whether I'm breaking my exclusive contract or not.
This line being iffier if I'm getting paid by a competitor in the field for the interview.
I would not apply. Besides the fact that I don’t want to spare half a working day unpaid, I find it worrying that implementing a feature on an unknown code base without any kind of collaboration with co-workers before or during the implementation step is considered normal in that company.
I’m not a fan of pairing all day, but coding tasks like these always seem to assume that it’s normal to not talk to other experts in the team, and that asking for advice or brainstorming on solutions or pitfalls is the exception.
> that implementing a feature on an unknown code base without any kind of collaboration with co-workers before or during the implementation step is considered normal in that company.
Isn't this collaboration? They've identified the codebase that needs changes, so it's not unknown to the company, just to you. They've designed the feature. They'll review your work in 3 hours.
I don't want to work at a company where it's not normal to get simple half day tasks with enough detail that you don't need further communication. I want to work independently, not be stuck communicating all day.
as a rule of thumb, I think the person doing the implementation should be part of the analysis. And as a second rule of thumb, I think it should be avoided to let someone dive into a completely unknown code base alone, at least when we talk about implementing new core features.
I’m not advocating communicating all day, but I think coding exercises like these miss the very important collaborative aspect of coding.
I would also not try and find a drummer for my band just by listening to them drumminggor three hours, instead I would like to see them performing a bit with other band members.
I feel like this type of interview misses otherwise good candidates who struggle to perform under this type of interview pressure.
Granted that they would be generally undervalued in interviews, but for me a truly great interview question/interview setup is not about a clever technical question, but rather about being able to assess true on-the-job ability despite the interview setup.
That's because 99% of people you hire are hired not for their ability to provide crazy insights, but to write bug-free and performant code with reasonable speed. "Here's the spec, here's the repo, can you get it done by the next release cut?"
The best received interview question we used to give was for a role that involved fixing existing code bugs. The technique to design such a question: distill an actual problem your team faced recently into something a competent developer could solve in about 15 minutes.
The problem: we had a credit reset script that users complained about. Apparently it gave incorrect resets. We sanitized the db tables needed and introduced a couple more simple bugs in the data, sanitized the script and provided a couple different language adaptations with a couple added logic bugs.
The candidate was given access to the repo and it had a clean way to reset the tables. The candidate had 45 minutes to fix the issue in the associated bug report tickets.
We received so many compliments from candidates, even those who were not hired.
TL;DR: design a work sample test and give that. Base it on recent work. Give the candidate 3x the time to solve it than what it takes you or a midlevel dev on your team to solve.
This can be optimised why only use problems that the team already solved? Just give the candidate access to the current JIRA tickets and let them choose one to tackle. Think about all the value that you could be extracting /s
> This is a great engineering interview problem because it pretty cleanly partitions the candidate pool into three different types:
> Type 2 looks at the problem and says, “Ah! I know just how to do this! Multiplication is just like addition, except wherever addition does +, I should do .” So they copy-and-paste, change all the +s to s, and they’re done in 90 minutes. Candidates in this group stand a really good chance of being hired.
As a non-expert in C/C++, I would definitely be of type 2, but I would be too afraid to apply for a job to work on a database engine's codebase.
I'm definitely not fluent in C/C++ and I think I might have done ok with this question, even if I probably wouldn't be a right fit for this job. I guess I wouldn't have gotten to this point in the interview process so it's ok.
It's not a bad challenge question.. that said, Is ask if the input and result should still be treated as an integer, floating point or arbitrary decimal value. What level of precision and if rounding at .5 should always be up or random in the case of interest precision.
For the simplest path I'm pretty sure I could figure it out. It's been a few decades since I touched C though.
Aside: currently job hunting and really hate the automated coding challenges. They all suck with too many assumptions. I'd much rather be tossed into a task like TFA in a language I don't even know.
Implement a gossip protocol for cache replication would be a good question, which doesn't necessarily require modifying memcached and would probably be easier and cheaper to maintain. Use case might be something resembling a TLS session store, even though we don't necessarily do that specific task anymore.
This comment is about some of the replies here. On the one hand everyone seems to agree that software engineering interview process is broken because it doesn't make sense to ask comp science implementation questions for an engineering job. On the other hand some people balk at having to do a 3 hour engineering interview...or take home tests or full day meet-the-team style interviews. Payment for interviews is a can of worms w.r.t legality, either that of you restrict your candidate pool to people not currently employed.
To those who refuse engineering intervies that take time - what is your solution?
However, as technical assessments go this is very reasonable.
It’s not them getting work done for free.
It’s only very minimally contrived.
It’s not “leet”, as in it’s straight forward (you’d essentially copy and alter an existing feature) and absolutely the kind the task you could encounter during a day job there.
If I were interviewing you and you were not referred (referrals might be able forego assessments) and if you refused to do this, I would feel that your attitude was a bit unreasonable and it would make me think less of you as a candidate.
But really it’s a willing buyer willing seller scenario, so you do what you feel is important to you, and I would do the same, but don’t be surprised if it counts against you.
> If I were interviewing you and you were not referred (referrals might be able forego assessments) and if you refused to do this, I would feel that your attitude was a bit unreasonable and it would make me think less of you as a candidate.
Don't worry, I would withdraw my application.
> But really it’s a willing buyer willing seller scenario
Yeah and my point is that I'm the seller and so should be every other candidate.
Without a strong referral beforehand, it's really you who doesn't need to worry, there would likely be no need for you to withdraw as most employers are apprehensive about making the financial and time commitment to an employee who is not willing to do even a reasonable assessment.
I was thinking on the lines of Type 1 myself -- some reasons I can think of in hindsight:
1. I assumed that low-level code and performance optimization was not the ask (given it was a mere three hour programming challenge with a unfamiliar codebase)
2. The very elaborate description of Incr, Decr with emphasis on atomicity and multiple clients -- was heavily leading us towards thinking about how to utilize the already implemented, stable, and presumably finely performance-optimized functionality -- rather than re-implement your own.
So I assumed it was a test of design thinking where you can apply the concept of atomicity, parallelizable operations, resolving conflicts / locks / race conditions etc.
If they are looking for talent in Type 2 space where they can do incremental edits to a codebase powered mostly by copy-paste -- I am not sure this intervew question is for roles that need the highest levels of generic problem solving needed from experienced Software Developers who will drive software design.
Also if implementing Mult was so straightforward that it can be done so thoroughly (including tests) in under three -hours by a single person that is opening the code base for the first time; Then why had this feature not been implemented already by then by the original project maintainers? I am sure I can find some use cases where there is a need to do multiplication of an in-memory numeric value say in gaming, cypto, etc. in order to justify this feature addition.
I think it's useful to have an interview question to identify engineers who, when faced with "add 'multiply' to a codebase that already supports 'add' and 'subtract'", conclude that it's an unimaginably complex design problem requiring the highest levels of generic problem solving. Lots of teams are looking for people like that, but not all of them are.
Agreed. Usually there is a lot of course correction that happens when there are at least two people / two brains working on a problem and bouncing off ideas. Unfortunately that is completely missing in such interview scenarios.
Hiring Type 1 is not a mistake unless that person is also rigid / headstrong and always goes for an overengineered solution.
I'd also assume that Mult can lead to overflow very easily, and if I'm not wrong, the solution proposed also doesn't check for such overflow/wrap around.
So I'd also concur that someone who blindly copy-pasted a "+" and changed everything to "*", doesn't really understand the difference between + and *.
That was my 1st thought. Traditionally the mul CPU instruction uses two registers for the result, as the expectation is that it overflows. Add/sub needs only a single overflow/signed flag.
OTOH, it's 64bit and only unsigned, so yeah 64 bit gotta be enough for everyone (unlike 640KB mem)?
Yes, I mentioned in a sibling answer - the mul overflows much easier and detecting is much harder, hence it uses two registers in the CPU for the result. Detecting sub/add overflow is a single CPU flag.
> Also if implementing Mult was so straightforward that it can be done so thoroughly (including tests) in under three -hours by a single person that is opening the code base for the first time; Then why had this feature not been implemented already by then by the original project maintainers?
Right. So if interviewers come up with an artificial contrived ("leet") problem, then candidates complain that it's an artificial contrived problem. When they come up with a realistic straightforward problem, then candidates complain that it should have been solved already.
Why would you automatically assume there's a clever trick instead of breaking the problem down and making assumptions like an engineer?
Very odd to me that you wouldn't see the question and think "multiplication is a pretty similar operation to addition, let's find the increment implementation and go from there".
Idk becasue I have some kind of trauma from interviewing I guess. And there is no clear description of what is being tested for with this test. I would have done multiplication in terms of addition since the code exists or look up how multiplication has been implemented by other projects.
Imo what he ends up calling type 1 is probably more along the lines of good engineering instead of copy and paste,which I would consider poor engineering.
Again it's not clear what is being tested for though.
It's pretty clear to me what's being tested: do you have strong fundamental engineering skills.
This kind of problem/situation is a very common pattern. Honestly it sounds like you don't have good engineering intuition.
Type 1 engineers as he described them aren't great engineers because they tend to get stuck in analysis paralysis. They don't have good enough intuition to make the right assumptions about the problem that allow them to continue moving forwards.
Do you really need to dig into the e.g. locking mechanism? Probably not, because it should be abstracted away or be used/implemented in the incr codepath, which you can copy.
Why are you so scared about implementing new multiplication functionality? Do you not trust your ability to figure out a robust solution based on addition?
In my experience, Type 1 engineers do not tend to break problems down correctly. You need to break things down so that you can validate/invalidate your assumptions. Having vague concerns that a multiplication implementation might break tends to belie a lack of understanding and trust, since otherwise you would either have specific concerns or be able to use your intuition to make assumptions about the system.
"good engineering" is not about going through every detail of a system, it's being able to reason about problems, approaches and solutions coherently and pragmatically.
Agree to disagree. You'll never convince me that copy and paste is better then reusing existing code or that it's considered good engineering.
And ime, having a job hacking on Memcached, things like considering locks are extremely important for performance. And a project like Memcached, which is a general purpose network server, meaning it used with a large variety use cases and in different settings and also needs to be consistently fast, you need to carefully think through changes.
So again this question is poor imo, it's not clear what you are actually looking for with it. Do you want to see me write code that compiles and does the bare minimum or actually talk about designing a robust feature.
> You'll never convince me that copy and paste is better then reusing existing code or that it's considered good engineering
What's better engineering: not solving the problem, or solving the problem in a sub-optimal way that can be improved upon in future?
I don't disagree that performance is important for memcached, but your base assumption here doesn't help you make any progress. You're basically giving up from the start.
Assuming that everything you need lives in the incr/decr functionality is a reasonable assumption that lets you make progress.
If you assume that performance is important and a multiplication operator may have implications, you should be able to come up with assumptions/hypotheses about why that would be and try to prove or disprove them.
> Do you want to see me write code that compiles and does the bare minimum or actually talk about designing a robust feature
The great thing about this question is that it forces you to be specific. It's very easy to make general statements like "we need to consider performance implications" without having anything to back that up.
Why is the bare minimum insufficient here? Why do you need to deviate from the existing implementation and re-design this operation in a robust way? What makes the existing implementation unsuitable for this use case?
Those are the kind of questions I would expect to be answered by someone who is concerned about the performance implications of a multiplication operator.
I have found that strong engineers do not pose these vague, open-ended questions and concerns without being able to dig into the underlying details and be able to validate or invalidate their hypotheses.
You can (and should!) ask that question if presented with this problem. From the presentation in the article, it seemed like they were looking to see code the compiles and does the bare minimum.
From my hiring experience, it can be pretty hard to get bare minimum from candidates, so I've never looked for more beyond that, but I think some of the other people on my teams have looked for more in their interviews. I would rather hire someone who can write code that does what we agreed (or really, what I said) it should, but isn't great at thinking about robustness than someone who can think about robustness but can't write code that does what we agreed it should do. I've worked with lots of people who weren't great about robustness, and have a handle on how to get them up to speed. I've worked with a few people who couldn't code even though it was part of their job, and I don't know how to get them up to speed in a timely manner. I have helped maybe one or two people go from a non-coding job to a coding job, but it's a long process, and I shouldn't need to do it for someone who is supposed to know, so it needs to be tested for in the interview.
While I still think it's a good way to assess how a person works, by actually making them work, I do agree that there's an issue with people thinking that there's some trick.
We've just completed a series of interviews and everyone below a certain age seems to think that we're pulling all kinds of tricks. During our interviews we'll ask a series of fairly simple questions about programming, operations, deployment, monitoring, all sorts of things. It's pretty clear to me that some candidates completely overthinks the problem. This was for a number of entry level jobs, so expectations aren't all that high, but we do want to make sure that people know the basics. It completely throws of a large number of people, especially those who have worked in startups and larger companies in the Silicon Valley / San Francisco areas.
Our older candidate frequently just straight up answer the question and just comment "Or where you looking for something in particular?" Normally we're not.
This is more a result of absolutely broken interview processes at other companies. I don't see us changing our interview process to include trick questions. Our questions are intended to make people talk, that's it.
