My current company used a "homework assignment" and they bought a lot of my trust with it.
1. The "assignment" took me about 4 hours, which they expected and told me off the bat.
2. It involved mostly relevant code with 1-2 wrinkles that were not odd just not everyday sorts of things.
For the proceeding interview, I was able to sit with 3-4 engineers for 1-2 hours really just diving into details about my experience and their processes, etc.
So if I could offer advice it would be to find a "homework problem" your existing crew could build independently in 4 hours or so. As in you've actually had more than 1 team member attempt the request, and they did it in about 4 hours each.
This was actually an interview process I didn't mind. I have 15-ish years of experience doing development. I can spare half a day for a prospect no problem. It was actually kind of fun.
4 hours seems to be a good "sweet spot". You can ask something sufficiently technical and oriented to the daily work without being a bore. And when the interview comes around, you can make sure it's not some ego-bound contest, more of a conversation.
At my previous company, I was the hiring manager and gave a homework assignment that was targeted a little shorter than you describe, at right about 2-3 hours. I was clear upfront that it would take about that long, though if you wanted to take longer, you could. Within reason, I didn't hold the time against the person if they took longer.
The crazy thing was that over half of the applicants that passed a resume screen and phone interview could not complete the homework assignment. Over half. Some couldn't get it at all (either gave up or had compile errors), some got it to run but it didn't work, a few got it to mostly work but their programs had serious bugs, and the last few actually got it right.
The assignment was to read a file containing a list of numbers (some formatted incorrectly, so there was some very simple parsing logic involved), call an API using each correctly formatted number as a parameter, and store what the API returned to a file.
I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
The last two Homework assignments I got took about 50% longer than the hiring person suggested they would take.
I'm perfectly competent at writing complex code. In fact I'm usually the guy who fixes bugs or perf issues that nobody else believes exist.
A problem you came up with on your own is going to seem simpler to you than to everyone else. Why? Because you thought you picked it at random but your subconscious picked the one it thought was good. In other words, you picked the one you were already primed to answer, and it seemed simple.
So either people are going to work at it longer, the timid will give up (there's nothing wrong with timid coworkers as long as they have someone to listen to them), and the brash will get it done in the time allotment by cutting corners.
Are those the people you were trying to hire? Because that's who you're gonna get.
I'm sorry but the problem is a trivial one. Even if you've never read a file, never made a http call in code or written to a file, these are all things a 5 minute stack overflow away.
I absolutely would not hire someone who failed that test. Your defence of it seems bizarre to me.
chillydawg, I take hinkley's assertion to be more of a warning to be careful about biases that may have inadvertently been introduced through the testing methodology. I think it's impossible for us to know without seeing the test you use, but it's always possible that the requirements aren't as clear as you think they are, or potentially what you deem as trivial may be biased from recent direct experience on a problem that you've then turned into a test.
I think these tests are a great tool, but like anything we all just need to be aware and careful of the biases introduced. Also, we need to be mindful that really good developers who are employed, have families etc, may not have an over abundance of time to spend on these, so if 10 companies each send a challenge that takes 4 hours, and it's unclear if the assessment will lead to further interviews, some better candidates may give up on the process or be selective about the ones they complete.
I think this creates an additional bias, that the people who complete your assessments may be the ones with the most time to invest in the process (ie the currently unemployed).
At my current job, we have an "onsite" challenge that we used to hire some Cuban devs. We went to Cuba (from Mexico) and gave the candidates a codebase for Tetris in an interpreted language they did not know. They have 2 hours to transform that Tetris into a Nibbles (snake) game.
The catch? There is no internet in Cuba, so they had to do it using a book, and asking questions to a programmer who was helping them there.
I think this is a good point. There's a real risk that a homework you think is simple is a little more complicated because you're making assumptions. I think it's good to have a few current team members actually code it up (not just whiteboard it), and also to talk with a few candidates to see how long it took them, to calibrate this.
It's also better to aim small (a la FizzBuzz) and walk up the complexity slowly, as opposed to starting with "Solve this ticket from our prod JIRA queue, with tests please" (which I have actually been given as a test!)
I like to aim for a very simple task, like proposed above, with maybe one non-obvious part or edge case (which the spec mentions, so it's not a trick question). Just enough to demonstrate a) clean coding for the simple parts, and b) ability to solve something without using a completely naive solution.
You'd be surprised how many people stop looking at the perf numbers once all the tall tent poles are gone. They have no idea or tools for digging beyond that. I've worked on several projects where the customer still wanted the app to be 2-10 times faster (or worse, a competitor was 2-3 times faster) and people look at the charts and announce that they've done everything that can be done. Some hide in plain sight but others are just putting in work. For instance code duplication can hide a tall tent pole because it's chopped into five pieces.
I had a shared library running on a bunch of systems. One was VxWorks. Cross compiled. I knew nothing about VxWorks except that the code needed to run about 8x faster. I got about 2.5x out of my code by improving locality of reference problems that appeared to be intrinsic but just required experience to see. For the rest I filed a bug report to WindRiver and five to the cross compiler writer. At the end of this process I knew more about the VxWorks administration than anybody else in the group.
People give up, even when there's a clear business case for continuing.
Often the assignments are more complex than described. I wouldn't be at all surprised if this was the case here. Also 2 hours is actually a really short amount of time, it can take 30 minutes or more just to understand what the person is looking for if the instructions are not clear.
Okay. Now you're sitting with an interviewer to talk about your home asssigment. Turns out there were some changes in a system and you program needs to evolve accordingly: actual format is protobuff struct and API is asynchronous — you send a bunch of tasks, server queues them up and gives you ids, after some time, results are pushed to a client as they come up. You must store data in SQLite and account for HTTP or REST errors (retry logic, etc.).
Your one liner is neat and all, but now you have to do 2-3 hours of initial work + w/ever time it takes for adjustments in 2 hours scheduled for an interview.
You could've thought out this toy architecture up front with initial simple implementations or at least discuss what they expect you to provide, but instead you wrote a shell script for some nefarious reason, even though you saw obvious disconnect between estimated time and time you actually spent.
Nah, i'd hire the guy with the 1 liner over the guy with the robust architecture any day. A robust architecture is great and all, but I think it should be built after a need for it has been proven.
When I was a junior developer, I was just trying to get code to work. When I learned design patterns, and had some bad experiences, then I started architect "robust" architectures for everything. After perhaps 6 or 7 years of doing this, I started to question my insanity. Sure there were several instances where I was like "Damn, really glad I built it like that" but there were an equal amount where it just ended up turning a simple problem into a complex problem. There's no good excuse for that.
So now, I go high school simple first, and college hard later. If it means I have to do some rework, fine. I'm okay with a little rework, because in the end it aggregates to a simpler platform that took less time to build.
This is ridiculous. There are so many changes in requirements there (asynchronous with push results being the biggest), that everyone is going to have to throw away what they wrote an start over.
Anything beyond 10-20 lines for the solution to this problem is pointless levels of abstraction and would be a huge red flag.
Sure, but imagine all you knew was C++, which is true for many people. Several hundred lines of code right there. I'm a hiring manager, given questions like this and seen that exact result. Remember that many people cannot solve fizzbuzz.
Then your choice is whether you want to hire people who only know C++. For some jobs, you don't. But if you do, you need to have multiple tests and choose one based on the candidate's skills.
We have an in office test--the dev sits down at a laptop with internet access, gets clear instructions and scaffolding code. We know that the tasks take n minutes because we see the time that candidates take.
This. If people didn't lie (or stretch the truth) or cheat, we would only need resumes to know whether someone can do the technical aspects of any job.
Also, credentialling. Even if you can't do the work and only stay a few months, Google or Goldman or another big name on your CV will still open many doors.
In Germany there is a culture of not changing jobs very often (at most every few years, but even this should not happen too often). So if you have a "well-known" name such as Google or Goldman Sachs on your CV, but worked there only for few months, it is a clear sign that you were fired and about every future employer will be very cautious.
Outside the Valley and startup culture most companies looking for software developers in the US view a tenure of less than two years as being short, and a series of them indicates "job hopping."
I'm not commenting on whether that's a good or bad thing, because good or bad it's simply the way it is.
1. What is most important in your CV is the last employer. If it seems that you were fired, this can cause problems with getting a (decent) job afterwards. If you were poached, it is because the other employer wants you badly. So in your example your new "last" employer becomes JP Morgan or Facebook. So this kind of problem does not occur here.
This does not contradict the fact that it still leaves/can leave a bad impression on impression if your were willing to be poached without a very good reason, since can be considered as a strong sign that your are not loyal to your employer and only hunting for money.
2. Poaching is rather uncommon in Germany. As I outlined in 1 being willing to be poached multiple times is a sign that you are not very loyal to your employer. Thus there is a strongly reduced incentive for employees to be poached.
Also for poaching to work, you have to offer lots of money - I'd be really surprised if there are lots of employers that are willing to start such an "arms race with money". Why not simply offer decent money in your regular job advertisements and get good people without any arms race?
Poaching might make sense if there you want to get "rockstars" that are not available on the "free job market", but can only be poached. But I'm also not aware of any industry sector in Germany where there exist employees that are treated as rockstars. :-(
Your vision is clearly limited to Germany, where poaching doesn't exist because there is no jobs and no strong companies competing.
Just imagine a random company in a tech hub. They host a meetup once... the days after the meetup, most of their employees have been contacted by recruiters and other companies who attended.
If they want anyone in particular and/or if any employee's salary is not on-par with their value, here they go! :D
This has happened at my company. We give a similarly near-trivial hw problem, and sometimes people come in and literally "cannot remember" any single thing about the assignment, including the names of the columns in the data or the approximate objective of the HW. This is after performing plausibly well on a phone screen etc.
From my recruiting experience people cheat even during live-coding interview. I was pretty amused seeng a person whispering to a buddy outside of the video pickup area trying to get _any_ code written.
I'm going to assume "couldn't accomplish the task" means couldn't get it entirely correct. Do you provide a testing baseline that determine the expected behaviour? With TDD being the norm and correct way to test your program behaviour, I'm absolutely not going to go through every edge case with a fine tooth comb to check your "correctly formatted number" calls the API as you expect for a homework assignment. Provide a testing baseline or clearly demarcate the various possible edge cases as would be discussed in real life.
> I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
I once asked a candidate to write a for loop in C#[0], using my laptop and Visual Studio and watched for a solid 5 minutes while he struggled -- And I don't mean jitters around syntax, I mean, failing to declare a variable properly and while he knew what a for loop was, he had no idea how to write one in C#. His resume touted a series of projects that we estimated put him a little above a Junior developer and the only language he referenced was C#. I don't normally do these kinds of "on the spot tests" because I, personally, hate them, but I was cheering when I read Coding Horror's "Fizz Buzz" article[1] on the subject.
This was the most foul example and my only experience is with developer and ops/security positions, but if you're getting half of developers who are in the same time zone as the title[2], you're doing well. Does this happen in other parts of the business world? Some of the folks we get in would be like a Walmart cashier applying to run a finance department because "they've handled money before". And what if they got the job, do they think they'd come close to being successful? I'm all for the "fake it until you make it" and I'm very willing to overlook weaknesses around experience/education if the candidate is really passionate and I can get enough of a comfort level that they'll be a quick study and love the job, but you have to be realistic about your deficiencies.
A lot of the problem, I've found (being on both sides of this in the past) is with recruiters, as well. They tend to operate on a "throw darts at the wall, blindfolded" approach, with the assumption that if they send out a million candidates for a million jobs without regard for qualifications, that they'll get lucky, one will land somewhere and they'll get a commission from sheer volume on bad odds. This serves nobody well -- it causes companies to put up multi-step gates that involve people who aren't close enough to the field to discern a good candidate from a bad one via resume performing a filter operation that disproportionately disqualifies the really passionate but "not quite a perfect fit on the qualifications side" and it causes job seekers to take a bunch of interviews for positions they won't like/aren't qualified for and won't land -- causing them to have a miserable experience interviewing (and probably resulting in some dropping out of the job market entirely).
[0] The exercise was, specifically, write a for loop with an int value that is incremented by one, adds that int value to another int variable, and multiplies the result against the counter, assigning the value to a third variable. Print the result to the console. This was before "Fizz Buzz" became all the rage, I did my best to give a blindingly simple problem to just validate that the person could do something in the language.
[2] Never mind the details "Must have 4 years of experience in Angular", nonsense -- a topic for another post, but the way we write job requirements is about stupid. I mean, I've seen experience requirements that exceed the length of time the technology was in existence, and I'd rather see a candidate demonstrate an ability to adapt and learn new things than be overly concerned with whether or not they've used a framework that is really similar to a bunch of other frameworks.
Bad jobs are constantly in need of filling: people quit very frequently.
Bad employees are constantly in need of placement: people get fired very frequently.
If you're hearing about a particular job or candidate, the odds are they are bad. Most jobs are filled internally or by referral. IIRC, it's only about 20% of jobs that are filled by recruiters and/or cold job applications.
You're right about bad employees, but I've not found it difficult to weed those candidates out. At the same time, I don't fault a candidate for being laid off provided they have a good explanation[0] and/or they're otherwise a good fit.
As for bad jobs, even bad jobs need someone who is functionally capable of doing them. Sure, you have to be vigilant about filling those positions, but it's rare to encounter a "bad job" in this field that would be fine to have a terrible candidate in.
Regarding jobs being from internal candidates: It's interesting, because I hear that statistic rather frequently and often wonder where it originates or whether or not it has the meaning that is often attributed to it. Again, pulling from my anecdotal experience, though my resume was passed in from an internal employee for the two jobs I was most interested in[1], in one case my resume was handed to a sales person who reached out to me with a question about the company I had just been laid off from -- I had never talked to him or met him before and he lived about 100 miles away and in the other case, my sister handed my resume off to a friend of hers who worked there[2]. This friend and I met for the first time on my second day of work there. I don't even know if the guy is a software developer or not -- and that's the crux of the issue. If the recommendation is coming from someone doing the same/similar work on the same/similar team, that candidate is probably a good fit. If it's from someone doing wildly different work or who has no idea what a good candidate would look like, it's not much better than the blind solicitation from a recruiter. This is further complicated by the fact that many/most companies grant a referral bonus, incentivizing staff to solicit and (possibly) lowering the quality of candidates coming through that channel[3]. I'm really curious about others' thoughts on this, particularly those who've hired for their own companies.
The last issue with the 20% rule is that in companies that are dedicated to filling a diversity standard of some kind, it's hard to get there from internal employees unless you already have a very diverse staff. Reality being non-ideal, you tend to hang out with folks who are similar to you[4].
[0] There are plenty of reasons people get laid off/lose a job that have nothing to do with the competency of the employee. Companies go completely out of business, shedding all jobs or merge with companies where the whole department gets eliminated. Less "great" excuses are role elimination, though, this happened directly to me in January but I was offered a position for every job I interviewed for, so given appropriate context it can be fine as well.
[1] I interviewed at four places, all four offered me a position, two of the places were the highest on my list and I took one of my top choices. The two that were high on my list technically had my resume passed in from someone who already worked there.
[2] That it was my sister certainly helped -- she's been the CFO of a very well known global company and is currently the CEO of a mid-sized company -- her recommendation is trustworthy and people who know her know that she wouldn't embellish anything just because of family relation.
[3] I'm on the fence about this, though, personally. On the one hand, I'm less likely to recommend anything but the absolute best candidate for a role at company I work for if there's not a financial incentive -- it's nearly all downside otherwise. If the candidate works out, it's unlikely anyone is going to remember you sent that resume in. If the candidate is a mess, you get a ding on your record for recommending a fool and a subconscious ding to your own reputation that you associate yourself with fools (unfair as that is, I've seen it). If there's a financial incentive, there's always the "well, we paid him to do that; maybe he really needed the money" option which is bad, but not as bad. I got 11 people hired in at CompUSA when I worked there in my teens because they paid a $250 bonus. I had one really bad (as in, he stole things, and damaged things in fits of rage bad) recommendation that surprised my managers and me (he was a close friend and I had no idea he turned in to lunatic once he was being paid to do something).
[4] Though this does apply on racial/gender boundaries, one of the goals of diversity is to also avoid group think. Taking race out of the picture, you favor people who are like/agree with you and if the goal is to seek people who are hard working/passionate but not cut from the same cloth as the rest of the team, an internal recommendation will miss that more than it will hit it.
Hiring at the company worked like this, provided the resume wasn't passed directly from a member of staff close to that team:
(1) Applicant applied through web/recruiter/e-mail. (2) Resume was reviewed by a generic hiring team who covered every kind of job the company had and none of whom had even the most basic understanding of what a developer's qualifications should be[0]. (3) This same team does a myriad of phone screens to further whittle the pile. (4) Team's manager reviews remainder and often didn't perform a second screen since one was already done[1]. (5) Candidate arrives.
[0] These same people provided many of the "templates" used for the job descriptions, so you can imagine these were similarly bad. This practice resulted in the company, for years, refusing all resumes where the candidate did not complete a 4-year degree program. I pushed for an end to that for dev jobs after two candidates I hired in who had limited/no college experience turned out to be excellent employees.
[1] And in some cases, the manager knew nothing about software development and was put there from some part of the business, so s/he'd be about as effective at a phone screen as the hiring team.
What language was this in, out of curiousity? That timeframe seems very long for the task, but only because I tend to work with languages which have nice standard libraries.
We really need to stop pushing this HW assignment nonsense. It's offloading the cost of hiring on to prospective candidates. The asymmetry of cost makes it plainly exploitative.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400+ hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is, I should be the only one doing the assignment and there should be no wishy-washy rejections.
To add on to this, a take-home assignment should be more than just another filter. The companies I interviewed for with take-home assignments didn't actually discuss the assignments at all in the in-person interview - I spent the time to prepare for and do the assignment, at least pretend it matters.
That just seems odd. How were they checking that you were the one who completed them?
All of the interview processes I've had recently have checked I did the exercise by either asking me to describe what I built with no aid, asking me to walk through the code with it in front of me, or asking me to conceptually expand on what I built on a whiteboard.
They gave me the standard whiteboard interviews on-site after that. This was for a data scientist position. I did not get offered the job (but I accepted a good job offer a couple of weeks later, so whatever).
What if a company offered to pay you a day's wages, or something reasonable still (if it's less than that)?
Whittle down the pool even further and the ones you think look good on paper, and after an interview (or a series of them), have them come in, on a paid basis.
It'd be a sign of good faith that they're both serious about finding a good candidate, and respectful of your time?
I know some companies do paid stuff like that. If only it were more commonplace.
That sounds like a fair process. Employers are in a position of power and can easily abuse it. When the employer also invests in me as a candidate it shows they're serious about me and also ethical. Sounds like a win-win.
I still wouldn't if I currently had a job. By the time I get home my brain is usually pretty fried and that 4 hours probably means giving up a couple of nights.
It's a bit frustrating because different people react so strongly in both directions. I don't like live coding, but I'll be ecstatic with a pair programming session -- because I can evaluate them at the same time. Homework, I'm generally OK with, but I honestly hate it when you get a response like "We didn't like your code. Goodbye." On the other hand, I know some people who can't deal with the stress of writing code in an unfamiliar situation.
I wonder if the best thing to do is to simply offer the candidate a choice.
> It's offloading the cost of hiring on to prospective candidates.
Why is that a bad thing? The prospective candidates aren't entitled to anything, the burden should be on them to prove they have any capabilities for the job.
> What is the employer putting up? Generally nothing.
Totally incorrect, have you ever had to hire somebody? The initial phone screens alone to weed out the 99% of complete incompetents takes many hours/days. Even getting to the list of phone screens, involves many hours of sifting through resumes, where each resume is cleverly designed to hide the fact that the prospective candidate is incompetent.
This reeks of entitlement. If you want the job, you'll find four hours. They don't owe you a job and they don't owe you anything for going through their hiring process that they have designed to find people to pay a salary.
I echo the grandparent's sentiments, myself, from my own personal experience as well.
So its entitlement to expect an employer to not exploit their position of power? Sure, I'll take that label.
>If you want the job, you'll find four hours.
But what happens when everyone starts having these 4 hour screens? Now its a 40 hour process to apply for 10 jobs in a week (and this is before any commitment on the part of the employer). It's not sustainable or fair. And yet, there's no incentive for employers not to do it. So we as job seekers just need to suck it up and be thankful for the opportunity, right?
I work at a small company in a remote part of an openly backward state when it comes to startups, and yet got two hundreds applicants to file a javascrip developer position.
After interviewing 10 (say two three hours spent on each between contacting, scheduling up etc) it was clear every unemployed with 'can use word' as skill boosted their resume and applied.
I'm sorry for the real developers looking for a job but a company can't dump 20 days work on chasing and weeding liers.
I'm not a fan of homeworks myself but this is what it has come to.
I'd be payng but that puts the company and the prospect in a very difficult position them towards tax laws and towards their current employer non competes, the company toward again tax laws and then there's the absurd of paying for contract work delivered that's 99% of times not up to spec.
Beside, the unwillingness to compromise smells itself and given the choice I'd never get a potential troublemaker in that only thinks time=money and screw everything else.
That little empathy for both sided of the problems sure is a red flag.
Edit: well unless it's a contracting job then it's fine.
This seems like something a 5 minute FizzBuzz-esque programming assignment as a pre-filter could easily fix. The goal seems to be a barrier that only serious applicants will even bother to attempt, and that's fair. But when that barrier then becomes a 4+ hour time sink for the serious applicants it becomes a problem. There are plenty of hackerrank style sites where someone can show a bare minimum of competency without costing either side much time.
yeah I think 4 hours is a bit too much, in the end you only need competency as a screening.
we used to give out this http://play.elevatorsaga.com/#challenge=3 and asking for solution to level 3 which can be done in a short time + is the first non trivial challenge.
of course solutions can be find in internet etc, but it's pretty easy to look them up if they smell fishy and it'd all come down crashing at the face-to-face anyway
it's also a very good exercise in real world problem solving and a source of endless technical discussion since there's no single good approach, especially with two elevators to schedule.
It's the power to get 30+ people to jump through hoops that only one person has a chance at being chosen. Even worse, a good 50-60% of those people have no chance out of the gate, they just don't know it. Encouraging people with little to no shot at the job to do significant work so make your job of filtering them out easier just reeks of exploitation. Come up with a filter that's minimal cost for those applying, fizzbuzz style problems seem suitable.
And what's wrong with having a position of power? You say it like it's a dirty phrase.
Can you, personally, pay me a salary? If you can and I want to work with you, I absolutely think it's fair to ask four hours of my own time to demonstrate I'm worth consideration for it. If not, then why does your opinion matter?
You sound like a Libertarian. I'm assuming all the advances in worker's rights we've had over the last hundred years are also "entitlement"? If not, then I don't see what's different about expecting an employer to not exploit prospective applicants by offloading most of the cost to them.
I don't "want the job" per se, I'm interested, which is a bit different and we'll need the interview process to find out if I actually want the job.
Furthermore, I already have a job, just as most reasonable candidates do - so if you want to convince me that I should spend many hours on that, you'd better be sure that your offer has a really attractive first impression and seems likely to be better than the job I currently have, otherwise you'll be self-selecting your candidates only from the (generally less skilled) sub-population of workers for whom "If you want the job, you'll find four hours" is true because they actually are desperate for jobs, unlike most skilled people in tech industry.
This phrasing may sound like entitlement, but try turning it around. Most of the people who will turn down a homework assignment are the ones who can afford to do so, because they already have a good job or know they can find one. Is that really the first filter you want on your candidate pipeline?
I have never seen a job posting I was sufficiently interested in to put in 4 hours with no investment from the company.
I have, and it's how I got the job I have now, which is by far the best job I've had.
Regardless, if you have a good job already, why apply to any position you're not particularly interested in and that you wouldn't spend four hours trying to get?
Because they contacted me and offered me to participate in the
recruitment process, and while I don't look for a job, I'm not against
checking where this can take us?
I've never taken a job I've applied to myself. That is, they were the ones who approached me (some more directly than others[1])
Why would I spend any of my already very limited time if they didn't make sure the process is streamlined?
Also, I don't think it's uncommon to like what a company does and being interested without yet knowing if you're interested enough to put the time aside - most company sites and job descriptions are much too generic and vague, so I'd want to talk to some employees first (remember, interviews are two way conversations). So I might not know if I'm interested enough at the time of application, just that I might be.
[1] for my current job, the interview process was essentially them trying to persuade me to join them, and my previous job I was actively looking for a job and someone I know at the company said I should apply. My "application" consisted of telling him "ok set up an interview"
I think it is worth seeing what's out there even if you're happy. Maybe there's something better!
The standard process where the first two steps are some ordering of a 1-hour interview with someone technical and a chat with a recruiter is perfect for this. With minimal effort (maybe 90 minutes) you find out whether they're interested in you at all, and get to ask both a technical person and a recruiter about the job.
The big time investment typically comes later, and I can easily bail if I decide it isn't going to be a compelling offer. I've done this quite a bit.
I'm glad the homework process worked for you, but my point that it doesn't seem like a good idea for the company stands. If the first thing you do is filter based on how badly applicants need a job, you're losing a significant portion of the best people right off the bat.
So now I'm doing a day and a half of unpaid work for them, taken out of evenings and time with the family.
Not hypothetical. I just described my last job hunt, 16 months ago.
The place I really wanted to go closed the req on me, and then opened it back up the day after I started at another place. My third choice (stars didn't align on the other 2 interviews, in part because of stuff like this).
Additionally, most of us are more willing to do interview work in our off time. Taking a vacation day to interview is a direct financial investment. Spending a Saturday doing a 4 hour home work is a sacrifice, but doesn't require taking vacation time. Taking an early afternoon off to go talk with 3-4 engineers for 1-2 hours is something that can much more easily fit into ones schedule.
EDIT: I want to provide an anecdote around this. If your salary is $150k, it may cost you roughly $400 in gross pay to interview somewhere. Sure, you could take paid vacation, but vacation time is paid out in cash when you leave so it still costs you. Companies with unlimited vacation are a different story.
Just wanted to say that I really like the idea of testing the "homework assignment" on your own team to hone it.
I had a similar experience a month ago. It was a similar "4 hour project" but it was open ended with "make it as complicated or as simple as you want". Being a single table backed CRUD app, you could only put in so many capabilities but it got me excited thinking about ways I could surprise the staff. In the end, I made it self-hosted (in a language that self-hosted is less common), included an installer, and made it support multiple connections from different devices with updates that pushed to the connected clients. The project took me about a day and a half (with two of those hours spent yelling at WIX), but the core parts took about two hours. I was genuinely surprised at how much fun I had putting it together. They also asked me to estimate my time spent -- since I had contract work I was doing at the time and couldn't dedicate a long stretch all at once to it -- I broke it down into its individual tasks and documented the whole thing. They had no problem with the fact that I delivered it a week after the interview.
I also second your "I can spare a half a day for a prospect" attitude. There's a flip-side benefit to that, as well. A candidate who feels over burdened with a simple assignment (provided it's tested as you mentioned and truly is simple) is probably not one I'd be interested in hiring, anyway. I'm not saying it's "one size fits all" -- I've had candidates walk in the door that I knew in advance were really good fits who were hired after a 30-minute screen. That, though, is a unicorn sighting.
What if the HW assignment became the norm. Could you spend half a day, or a whole day to 10 jobs? What if after you spend all that time you still don't get any offers, or even interviews? Job searches are already hard enough (for most people) without these huge time sinks becoming the norm.
That would be fair. But having that sort of expected result is impossible for many people. Of course, some people are good enough and sell themselves well enough that they'll never have problems passing such screens and getting offers. But for those that aren't, and worse don't know they aren't, will be able to reach that level of expected return without the process being designed not to exploit those situations.
Requiring a take-home assignment, not just offering it as an alternative to an in-person interview phase, is a great way to systematically reduce the diversity of your hiring pool. It works well for some, but please don't require it of everyone.
I am confused by this, could you help me to understand why this is so?
In particular, it seems to me that take-home assignments are strictly better for candidate scheduling etc than in-person interviews, if time spent can be held equal. Maybe time spent is just not equal very often?
Yes, absolutely. I say this every time this homework assignment shit pops up. It's offloading the cost of hiring on to prospective candidates. The asymmetry of cost should give everyone pause. It's plainly exploitative.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400 hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is to say, I should be the only one doing the assignment and there should be no wishy-washy rejections.
> What is the employer putting up? Generally nothing.
Unless you've been in the shoes of a hiring manager, I'm unsure as to where this idea is coming from?
Having been a hiring manager, it is a horrible experience. Unless it's extremely naive, homework sets generally take time to come up with and formalize, and certainly take time to review the results for.
I suppose I can understand conflating the motivations of an interviewer with their employer, but it's not generally true to say that interviewers are just trying to exploit those they interview.
>homework sets generally take time to come up with and formalize, and certainly take time to review the results for
It's a one-off effort to come up with an assignment. So the cost per application can be extremely low. As far as evaluating them, the bad ones can usually be tossed out in minutes or less. Comparatively, the cost per applicant is basically nothing.
How many companies waive the coding requirement or postpone it towards the end, if you already have open sourced a bunch of code. It is also possible that the HR managers are just happy taking a route, where they can offload most work to the other party. The can instead act as better intermediaries, aligning the hiring goals of the company and the experience for the applicant. Whats wrong with asking them to create better value or understand the domain better so as to be able to create better value?
1) How many forks and stars do you have?
2) How often have you opensourced?
3) Have you blogged about them or created a documentation?
4) Why did they build each of those repos?
5) Commit logs... etc..
Very easy to develop such a framework.
One or two repos can be copied, but there is ample signature on a genuine open source contributors.
> It's offloading the cost of hiring on to prospective candidates. The asymmetry of cost should give everyone pause. It's plainly exploitative.
There's nothing wrong with that, it's a competitive market, those who get hired are those willing to compete, not those who complain about unfairness about being asked to demonstrate their skills. Employers spend tons of time filtering candidates, even reviewing the submitted work takes time and effort, saying it's no cost to the employer whatsoever is not only untrue, it's naive.
> So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing.
What?! Up until that point, typically, the employer had crafted the job posting, posted in on various channels, reviewed the responses, talked to promising candidates, and then give you the assignment. And a good assignment needs to be carefully crafted, a process that usually takes an order of magnitude time more than what it takes to prepare an answer.
An order of magnitude is not much though, that's only ten applicants? Most assignments waste the time of hundreds of applicants, so it sounds like its still a net win for the employer.
It feels to me like the root problem here isn't how many people are doing the homework problem or whatever. It's that this is increasing your time spent as a candidate relative to the in-person interview.
I think your last paragraph is very close, but I think the thing that matters is "no wishy-washy rejections". Not being the only person taking the test or how much time the company spends seem like ways to make that happen to me.
Maybe I'm just completely wrong here, but I think most of the objections to homework type tests are being caused by processes where the homework test was in addition to the in-person whiteboard interview, not instead of it.
How would you feel about a process that was: some preliminary screening -> nominal 4 hour homework thing -> in person meeting to confirm that you're not a complete asshole and negotiate salary?
Job seeking is a sales activity. You sell your labor on labor market. Every sales activity has a cost. If you do not want to bear this high cost of IT industry, you can work at McDonalds. Probably your sales cost would be much less but your profit also.
It is a sellers market. I get recruited every single day. Any company that disrespects me and wastes my time isn't going to be very successful at convincing me to work there.
Developers are the ones with the power in this market, and I have no problem throwing this power around.
He is saying that the company needs to tell their people to fix the process and do what they are paid for. Not offload it entirely to the applicant.
Maybe just open separate channels for people who have an open source portfolio. I have about 30 repos in my OSS, but still have to take these assignments. Its such an utter waste of my time.
It does, if they're actually doing it in a way that's useful to them. At least one, probably two, engineers are going to evaluate what you send in, take notes, and then go over it with you. And they have to do that for every response.
So that's an absolute floor of an engineer-hour per candidate: Anything less than 30 minutes to read through it and take notes is useless, and figure another 30 minutes minimum for a call.
If they're not going over the responses in detail and then talking through them with the candidates, then they're just wasting their own time because they're not actually judging anything.
That's a good way to get passed over for good jobs. As a potential candidate, you aren't in a position to making such demands and your unwillingness to demonstrate your skills on a simple assignment just means you won't be considered. Plenty of other fish in the sea who have no problem completing a simple assignment.
It depends how desperate the candidate is for a job. A coding assignment may have a hidden cost of filtering out many qualified candidates, if qualified candidates don't feel the need to apply to companies with these assignments. Anecdotally, these assignments seem like just another filter companies use, they don't replace any steps in the process, just one more hurdle to pass.
If you aren't willing to demonstrate your skills with a little coding assignment, I don't consider you a qualified candidate. I've used a coding test for years, always get exactly the good kind of people I want, it filters out the lazy and the liars, most programming candidates are liars and simply don't have the chops they claim.
>Plenty of other fish in the sea who have no problem completing a simple assignment.
And yet companies have an incredibly hard time hiring talent, even with so many other fish out there. Odd, that.
People need to start understanding their value as an employee. You create more value than you're worth; that's why companies need workers. You don't need to bend over for a potential employer. Plenty of other fish in the sea of employers, too.
It's not odd at all, those are old school companies playing by the old rules of looking for candidates locally. And demonstrating your skills isn't bending over, it's showing your value and it's how you get a job. Seriously, this notion that you should be hired for a high paying job without needing to audition for it is just absurd; I wouldn't hire anyone with your attitude.
I get recruiting emails literally every day, and people call me on my cell phone, unprompted, once a week. So yes, I absolutely am in such a position to make such demands. If a company doesn't like it, then they can do me a favor by passing me over.
And I am honestly just an average SF engineer.
The current engineering market let's me choose whatever company I want.
So, I am not going to waste my valuable time with companies that disrespect me, when there are a hundred more that I could be talking to instead.
Great, but that's an artifact of living in the SF bubble which isn't anything like the rest of the planet. That is not the current engineering market, that's the current SF market. And asking you to demonstrate your skills isn't disrespecting you, you work in an industry that doesn't have a professional certification which means it's loaded with people trying to fake it for the big paycheck; due to that, it's perfectly reasonable for companies to ask for proof of skills. If that makes you feel disrespected, you aren't in touch which what it takes to run a business.
Don't care. I am the one with the power. And I have no problem using this power to my advantage.
The way that a business relationship works, is that the person with the power gets to make the decisions. And in the market that I am in, that person is me.
And I have no problem demonstrating my skills. I just demand that there is another person from the company, on the other end of the phone line or other end of the table, investing the same amount of time into the relationship as me.
If a company thinks that it is unreasonable for me to demand an equal time investment from them, well sucks to be that company, because as we have already established, they aren't the ones with the power, so their opinion doesn't really matter. They can go find someone else to interview.
Oh most happily will, even talking to you is a pain, I wouldn't work with you if you paid me. You're arrogant and entitled, you'd make an awful employee.
It may depend on the industry, but in tech generally there's not plenty of other fish in the sea - there's a scarcity of qualified candidates but plenty of other potential employers who desperately want to hire.
That's really not true; there's only a scarcity if you insist on locality. There's no shortage of qualified remote workers when you open up your search the whole country or whole world. Employers crying about scarcity are employers who haven't figured out remote working is the future.
If you're applying for a job, you're asking for the company to pay you a salary and to believe you can provide labor worth that salary. You can find a few hours.
I agree its not a process of applying for college, its a job. Testing to see whether somehow is a recent BSCS helps to systemically impose all 3 of the most problematic isms, racism. sexism and ageism. You can get brilliant employees from bootcamps, my current coworker is a truly brilliant bootcamp guy. He's African American and decided to skip student loans and go direct for code bootcamp and for front end he's really good. But if you tested him over recent BSCS type of stuff you would be screening out a brilliant front end developer.
I wouldn't hire a frontend-only guy instead of a qualified developer, for the same reason I wouldn't hire a guy who can only mount wall outlets instead of a qualified electrician. Everything he didn't care enough to learn is a risk.
If the homework assignment is at least somewhat related to the job (not CS esoterica, etc) then I can't really imagine a better way to measure the candidate's relevant programming skill, which is certainly not the only thing you're hiring for (if you're smart) but it's important and I don't really see how it reduces the "diversity" of the hiring pool in any undesirable way. Could you elaborate on your reasoning?
People have children or other life situations that make it harder and less desirable on average to give up weekends and evenings for a homework assignment. It optimizes for people more willing and able to use their off-hours for work, which is not an even distribution across the population or even necessarily a desirable trait in candidates for that matter.
Assuming the homework assignment is happening in the hiring phase after the initial screenings, I don't buy this. Getting to that point means it's time to show them why they should hire you, and it's going to take up your time whether you do it in person or "asynchronously". If they give you the opportunity to spend those couple of hours at home rather than at a whiteboard in their office, but you don't care enough to make the time, I don't know why they should care enough to hire you.
If the "homework assignments" are just a scheme to get work done for free, or a competent engineer's time is not set aside to review them, then that's a different story. But as a fair test of programming ability, administered in good faith, I don't see the problem, and such a test is important.
My advice to job seekers who are asked to complete a "homework assignment" by a potential employer:
Ask to see the company's employment agreements or contracts in writing first.
I was once given a 10 hour assignment, which indeed took that long to complete. Afterwards, the company expressed interest in bringing me on, and sent me a work agreement to sign, which included many unsavory terms such as being paid for completed work 90 days out.
Since their terms were shit, I didn't take the job, and lost 10+ hours of my life.
When I was hiring, I used a homework problem as well - something that we have actually built and released, and I encouraged candidates to look at our implementation (it was a single screen in a very UI-heavy mobile game). We did it in 8 hours, but almost all candidates took more.
Overall level was pretty low that after reviewing about 20 homework assignments, and I had wasted a lot of time on post-mortem interviews where I just had to politely decline with a short code review (after all, they spent all this time, so they deserve something in exchange). After all, I started asking FizzBuzz (yes, not changing the name or anything) on the first phone interview to save time. And yes, turns out, it's not a task that any candidate can do.
So, yes - homework assignments are great, but make sure that a candidate at least knows how to write a simple loop (and yes, I overlook off-by-one errors).
My current company does something similar. I think back when I had to do it, it took 2-3 hours. It's a pretty basic problem using strategies and observers. The next time we're hiring I'll probably update it.
The kicker though, like someone else below mentioned, is that we would receive submissions that didn't even compile.
What I liked about the HW problem is that even something simple gives you and the candidate something to talk about. Why did you do this or that. Tabs or spaces ;) etc...
I have not followed up with places because I was worried that the questions were representative.
Remember, the interview goes both ways. You can look dumb for your choice of questions just as much as the interviewee can look dumb for their choice of answers.
I know that "homeworks" aren't very popular on HN or Reddit. However, when job-hunting I have infinitely greater respect for companies who use them, rather than sophomore C.S. whiteboard exercises.
As imperfect as ALL technical recruiting styles may be, at least a short coding assignment (i.e. < 4 hrs) is halfway-related to the real world. I have written code against a set of requirements nearly every day of my professional career. Outside of the interview context, I haven't balanced a red-black tree since I was in undergrad over 15 years ago. Frankly, whenever I hear people favoring the latter over the former, I always assume that it's just because they're fresh out of school themselves.
Yeah, doing dozens of homework assignments for multiple companies could be a massive time sink. However, I've almost universally felt like I wasted my time when going through the interview process with companies who don't use them.
Perhaps my perspective is a bit colored, as I'm far enough along in my career that I can be choosy about which companies I invest time in. I only talk with a dozen or fewer companies in each job search, and so I won't do a 4-hour homework assignment unless I'm already interested in working for you. If I were closer to entry level, and therefore had to go through the process with a hundred companies to land an offer, maybe I'd have a different opinion.
Regardless, the original post here is some entrepreneur, writing a small novel about his lengthy process of figuring out which job title to seek at a real company ("CTO"? "VP"? "Architect"? "Principal"?) Of course it took him 200+ hours to get a clue.
If you're not willing to pay me on the order of $100/hr to do your homework, I'm not willing to play your recruitment game.
My time is valuable and if an employer isn't willing to give equal effort in the interview process, I'm not interested. Also I'm not particularly interested in a workplace – especially a startup – that insists on this in a recruitment process. It leads to only hiring employees who are willing to play the game which is much more likely to exclude people who want to live lives outside of their job and fosters a culture of obsessive overwork as well as not having the diversity of opinion/demeanor/background/etc. which is valuable in a healthy workplace.
If you want to bring me in and spend a day with me talking and working in your office, fine. I won't, however, spend 4-10 hours working on a problem for you while working at the job I already have and doing the rest of life.
The thing is, the companies giving homework assignments still go through the regular onsite interview process. So it really is just an added burden of stress and time on the candidates. Arguably studying CS problems also helps job performance, though probably not as much as it could.
That's not always true. The best interview experiences I've had were at companies where you were given a homework assignment, and then invited on-site to talk about your solution for that assignment. That's not at all comparable to the standard Apple/Google/Microsoft/Facebook process where you're expected to come in and solve some graph theory or dynamic programming problems on a whiteboard for 4-6 hours.
I would much rather do a homework assignment and then come in and have a code review or design discussion about my solution than do the standard process. And, I would argue that it gives the company a much better look at my skills and aptitudes than the whiteboard interview. I have never once written a dynamic programming algorithm from scratch, in 11 years of programming. I have implemented graph theory algorithms only a handful of times. But create a relatively basic piece of functionality, and then justify my solution in code review? I do that literally every day, and sometimes more than once per day.
I understand that writing code for a homework assignment can be seen as giving the company I'm interviewing for "free labor". But, as others have pointed out, from the interviewee's perspective, there's not a huge distinction between spending 8 hours cramming Introduction to Algorithms and Cracking the Coding Interview versus spending 8 hours making a website using Django or Ruby on Rails (to name two interview tasks that I've actually been presented with). Frankly, I was much happier writing the websites, as it seemed much less pointless than memorizing how to write Floyd-Warshall all-pairs shortest path in Java on a whiteboard.
Senior candidates are in the worst position for the typical algorithm/data structures interview. College is a distant memory and they need the weeks of study more than fresh graduates.
Exactly. As a 15+ year vet, I love coding homeworks because:
1) I've been coding forever, and have faced code reviews in multiple professional environments. So your "6-8 hours" will probably only take me less than two, and my code will probably look better than yours to a reviewer.
2) I get tired of having to re-read "Cracking the Code Interview", because almost none of that is frequently used outside of academia. And dealing with smug 5-year journeymen interviewers, who are accustomed to asking 1-year juniors to traverse binary search trees and name all of the design patterns that they've memorized from books.
No matter what process your company uses, there will always be people trash-talking it on the Internet. "I refuse to interview at those places", "I insist on being paid an hourly rate to deal with them", etc. I'm sure that 99% of this is chest-thumping bullshit. But regardless, if you're going to filter out part of the candidate pool no matter which path you take... I'd much rather filter those 25-year olds who feel too entitled to write some code, and legitimately believe that companies should just glance at their GitHub page and then hurl them a giant sack of money.
Ironically, one of the things outside interests can teach you about is something called overtraining, where you can actually get worse at doing something from doing it more or harder.
If you've heard cross-training in a fitness context, it's named that because it's intended to be an antidote for overtraining. Branching out makes you more resilient. Your ability to solve problems isn't helped by hyperfocussing on an extremely narrow context.
Any inventor can tell you that the really big stuff comes from fusing ideas from seemingly unrelated disciplines. And yet we bury our heads because we don't know any better, and we don't know better because we've buried our heads.
The difference is that you, as an applicant, want to be part of something someone else created and to use it to feed and clothe yourself.
Nobody is asking for free work. And I definitely would never hire someone that calls a four hour exercise that is part of an application process "an indication you don't value yourself highly." In fact, it would be a huge red flag and I'd throw that applicant'a resume into the trash can.
> Perhaps my perspective is a bit colored, as I'm far enough along in my career that I can be choosy about which companies I invest time in. I only talk with a dozen or fewer companies in each job search, and so I won't do a 4-hour homework assignment unless I'm already interested in working for you.
Interesting. I've found the opposite. The farther I get in my career (and the older I get--not going to say the "A" word), the more difficult it is to get through the interview gauntlet. Companies looking to fill Junior "5th engineer from the left" roles tend to be fine with a warm body and a brain. Companies looking for super-senior-staff engineers, engineering managers or even more senior folks are extremely picky and probably reject almost everyone they meet. Even when I'm not actively looking for work, I try to answer every recruiter E-mail try to reach out to 1-2 companies a week just to keep irons hot and be able to act on the off chance that something spectacular pops up.
The older you get the MORE shoe rubber you need to go through pounding pavement.
> As imperfect as ALL technical recruiting styles may be, at least a short coding assignment (i.e. < 4 hrs) is halfway-related to the real world.
Yes, but what it is not is economical. There's a huge dropoff in the interview funnel at this step because it's largely not worth the effort for the interviewee. The choice is between spending 5-10 hours per take home test per company or spending 5-10 hours a week studying silly whiteboard exercises that apply to dozens of companies.
It's also just as uneconomical for the interviewer. I've completed about a dozen of these tests over the last year and only one company bothered to look at the test after I sent it in. Which makes sense; it takes time to look at a lot of code, figure out which of it is boilerplate and which of it was written fresh for the test. And even then, there's no guarantee somebody else didn't write it. And while you may not be interviewing with hundreds of companies, most companies easily get hundreds of applicants per position these days.
For whatever reason, it doesn't seem to be worth anybody's time to do these tests. This bums me out because I'm terrible at live coding tests. But, this is the reality of the situation, so I have to adapt.
The way I see it a lot of the best people would probably never bother going through a take home test. A company with a simpler, quicker interview process would have already snatched them up.
Marginal cost to get some sort of commitment/investment out of a company is lower, though. With 4-8 hour coding assignments, that's ~10 companies per two-week period to consider you for the on-site portion. With all the other process stuff, you'd wind up maybe being able to get 1 or 2 concurrent job offers.
All of the take-home interview questions I've done have been given after a phone screen has already been done. I would think that's the only way to do it - it's going to take some time for a company to look over what you submitted as well. I don't think I'd bother doing one without that.
FWIW, I've done a total of 1 for each of the last 2 job searches I've been on, and ended up getting an offer out of each one, so it seems perfectly worthwhile to me to do them.
I think there are two main problems with software development recruiting:
1) Companies who are not Google or Apple like to read books about Google and Apple, pretend that they are Google or Apple, and imitate recruiting processes that come from Google or Apple. Which doesn't serve them very well, because they are NOT Google or Apple.
2) Candidates who are not "the folks who can walk out of Apple's machine learning department on Friday and into Google's on Monday" like to pretend that they are, congregate on Internet forums with other candidates who also pretend that they are, and exchange interview advice geared toward people who are. Which doesn't serve them very well, because they are NOT.
> Companies who are not Google or Apple like to read books about Google and Apple, pretend that they are Google or Apple, and imitate recruiting processes that come from Google or Apple. Which doesn't serve them very well, because they are NOT Google or Apple.
A million times this. I got rejected from a company because the interviewer decided he didn't like that I used array concatenation to find the correct solution to a problem he gave me. This was for a job at a 100 person company. Companies writing line of business software need to stop acting like they are google.
> Outside of the interview context, I haven't balanced a red-black tree since I was in undergrad over 15 years ago.
Which company asked you to balance a red-black tree in an interview context? Name-and-shame them. Seriously. I've heard many whiteboard interview polemicists use this example, but have never seen any evidence for it actually happening.
In the programmingzen blog, Antonio Cangiano, a ruby programmer and author of several ruby books, there is a post about an interview for a job at IBM in which the question is to explain how the red-black tree algorithm works.
what's to prevent someone from outsourcing the "homework" to programmers on the internet ? I interviewed someone on skype 2 weeks ago and the applicant clearly had someone off-camera in the room feeding them answers. People like money, and they will go great and nefarious lengths to obtain it.
> Use homework assignments to assess the candidate’s engineering skills
while homework problems can be better than in-person interviews, they can also be a lot worse. for one, because they are NOT time-bound. secondly, they are much more difficult to setup and evaluate: do you provide a spec? code to modify? third, asking people to do work for free in order to explore the potential relationship is a big ask. lastly, they are very time-consuming for the company.
in grad school i learned the hard way to always ask for in-class exams instead of take-homes :-). time and scope constraints can be favorable both for the company and the candidate.
> Enough with the graph traversal problems.
the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill. it is that they are a way to gain information about the candidate's problem-solving ability in a limited amount of time. unavoidably, these questions test for speed and familiarity along with problem-solving. that makes them a lot less than perfect, but i've generally found them valuable.
The issue with tricky algorithms questions is that they often become tests of how well you can keep your cool. I've (and I'm sure others have also) bombed relatively easy questions due to nervousness compounding. I've also done much better than expected on other interviews because I remained calm and kept the interview flowing smoothly. This is the real skill under test, IMO. Of course, your ability to solve problems is still important, and a subset of what's being tested. The issue is just that there's a lot of (possibly inherently random) factors that cloud that signal.
My favorite example of this is when an interviewer asked me about an algorithm. In the course of the problem it occurred to me, what I needed was to break down the input integer into two parts--two equal parts that could be multiplied together. I kept referring to these two parts as such.
After the interview he said I was clearly weak on math because I didn't know the definition of a square root. Something I did know (10 or so college math courses in linear algebra, calc etc) and was describing, but was at a loss for words.
> asking people to do work for free in order to explore the potential relationship is a big ask
Asking to work somewhere without demonstrating that you can do the work is asking a lot. Unfortunately, job history alone is not enough to weed out people that cannot program (let alone well.)
> they are very time-consuming for the company.
Not if you implement a simple test spec or scoring system. We're engineers!
> the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill.
So don't ask the generic questions. Ask someone to implement something that they would need to do on the job. Obviously don't ask them to implement a feature that takes 12 hours, but a basic screen that is a snapshot of 45 minutes of on-the-job experience isn't crazy.
We also do a take-home exercise first to see how you think without us staring at you. Day-to-day at Seneca Systems we aren't staring at you, so it feels more relevant.
> Asking to work somewhere without demonstrating that you can do the work is asking a lot.
Which other professions have this attitude? I've either worked or supported a pretty wide variety of other jobs, and I can't off the top of my head think of a single one that does.
Even if some examples are brought up (and I'm genuinely interested in what they'd be), it's far from the usual approach in most hiring situations.
Huh? Your answer surprises me, as I can think of tons of examples. Designers are often asked to provide a portfolio to show the work they are capable of. Same thing for photographers. In a previous life I was a ballet dancer, and the whole purpose of auditions is to demonstrate that you are a competent dancer.
I generally prefer an existing portfolio of code (e.g. github repos) to see the kind of code someone produces, but if you aren't able to show some fairly extensive examples of your work, homework is usually a bunch better assessment of your skills than what can be accomplished in a ~45 min interview.
Continuing: Imagine a guitarist wanted to join your band, but they refuse to let you hear them play until they're hired, saying they won't work for free.
Fair. I should've thought of designers and photographers. Ballet makes sense, since there are trials for prestigious orchestras and the like, but that isn't something I knew about.
When I wrote my comment, I was thinking of the medical profession, dentistry, auto & other mechanics, appliance and other device repair, construction ("there's a pile of lumber in the back, you have an hour to build a shed and then we'll consider hiring you"), plumbing, civil engineering ...
Most of those other professions you listed have much more meaningful and standardized certifications than software. A board certified cardiologist has already demonstrated real proficiency by going through the process to get certified. In software, the whole reason for these complicated hiring processes is that no one really trusts software certifications, and even company pedigree is not always a reliable signal.
Why is your company so special that I'm the one doing all the proving?
There are a lot of clowns out there, on both sides of the table. At this point I've had my quota of companies that sounded good in the interview process but ended up being like a bad second date. Full of chaos and not anything I needed in my life.
One place drilled me on testing, and I mistook that for caring about tests. Nope. It turned out that not only did they not have any test automation, but their code base was such a ball of mud that you couldn't have many of them if you wanted to. I had absolutely no interest in pushing for a renewal on that contract.
>>while homework problems can be better than in-person interviews, they can also be a lot worse. for one, because they are NOT time-bound.
Um, this is a good thing, not a bad thing. Research shows that stress significantly reduces cognitive ability, and time pressure definitely adds to stress - sometimes substantially. What this means is that when you ask whiteboard coding questions during the interview, you're not measuring the candidate's on-the-job ability. You're simply measuring how well they perform under severe pressure. If that's fine with you, as in it's representative of your work environment where people have to churn out code around the clock, fine. Otherwise, homework questions are much better.
Also the stress of someone standing over your shoulder watching you work. At work, I enjoy hashing out problems with a coworkers at a whiteboard, but in an interview it's paralyzing to have someone watching me. As a student, I'd totally freeze up in the middle of a test whenever a teacher started walking down my aisle.
i agree with all that -- again, the i don't think that whiteboard interviews are perfect.
but also, keep in mind that the candidate has a limited amount of time, and often a full-time job on top of the job hunt. the homework questions are only good if the candidate has time and desire to them.
I find this point interesting, as to me, this would be an argument for, not against. Time constraints on homework assignments are a lot closer to the constraints you'd have programming for a job than "whiteboard coding" would be. I'd hazard a guess that doing a good job on an in-depth, open-ended problem given a few days to a week is a better indicator of ability to be a successful software engineer than being able to solve a logic puzzle on a whiteboard in fifteen minutes.
If the interview time is paid, sure. But if it's not, agreeing to do unbounded unpaid work for a chance of being hired for a paid gig, especially when it's not even clear the chance is particularly strongly correlated to effort put in, gets worse as a proposition the longer you spend on it.
I was speaking more from the side of the company; from the side of the interviewee, it's less clear-cut; if you're the type that doesn't tend to do well in interviews, it might be preferable, but if you don't feel like you'll do a better job on a homework-like problem, then it's definitely worse.
By design. In the candidate pool, there will be some who see the odds as too steep. For others the cost of completing assignments is lower, making the proposition better. Not good, but comparatively better (less bad) than for the former kind. And that's how we know whom to hire.
i agree that it seems easier to simulate working conditions with a homework assignment. (it does add a lot of work for the company to do homework problems well, but let's leave that aside)
the downside is that the candidate has a limited amount of time to talk to you. they often have a full-time job going on top of the job hunt. secondly, are you really game to spend 0.5-2 days of unpaid work just for the chance of being invited to interview further?
Also mentioned in sibling comment, but I was thinking more from the perspective of a company trying to accurately gauge a candidate's caliber. From the side of an interviewee, it definitely isn't a clear win; the only case where I think a candidate would prefer this would if they tend not to perform well in interviews (which I don't think is necessarily a good indicator of whether or a not a candidate would perform well for a job).
An additional perspective I didn't think of until now is whether this type of recruitment process would be more beneficial to the industry as a whole. An argument could be made that we'd all be better off as a whole if interviews tended to more accurately assess candidates and candidates were able to get a more realistic sense of the type of work they'd be doing for a company.
Also it doesn't scale for the candidate. If you need 0.5-2 days of unpaid work for _each company_ and you're applying at 20 companies, you're spending more time applying for jobs than you would be working 10-40 hours of unpaid work just to get to the next step.
>the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill.
Actually, my codebase has an honest to god graph traversal in it, on the front-end no less. Basic manipulation of graph relationships without much friction should be standard.
They can absolutely be timebound, unique URL time stamped at the download, with however many number of hours from then to send in whatever you have, completed or not.
I've only experienced this once and it was stressful as could be, but was from a person I absolutely respected and still do. No-bs, either right or wrong and plenty to discuss in the followup conversation.
Did it from home, pulled the file at around 11pm right when I knew my best time to work was. Would happily do as part of an interview any time.
That's not my experience, being on both sides of it. (devops kind of tests) Given no bounds, you will not be able to produce a 100% solution from the recruiter side of view. Something will always look weird. There's always another approach, generic parts possible to extract, places that can be left available for future expansion, etc. I'd go as far as, if you can finish the test in 2h, then it doesn't matter if you spend 4h or 8h on it, because you'll just do more of the same in most cases.
For example you can document one module and say "the rest of the docs are TODO due to time" - it shows you know how to document and you can move to another bit. My favourite submissions did 5% of many things to show they are aware of those areas, rather than super-polished trivial solutions of just the basic question.
On the contrary, I'd say that having unbounded time tells the interviewer a lot about how you go about building things. Do you know how to build something that works and stop at an appropriate level of abstraction, documentation, and testing for the type of problem at hand, or do you go off into the weeds and build FizzBuzzEnterpriseEdition (https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...)?
> If you don’t fit their mold (“We need a Javascript engineer with 5 years of experience writing full stack consumer-focused applications.”), they won’t know what to do with you
This is the truth and it hurts. Take a company with 20 job openings, each of which you are probably semi-qualified for, and could see yourself doing. As a candidate, you're limited to applying for one of them, and if you strike out, you're back to square one--often below square one because companies make you wait N months before applying again.
I'd love it if a company were to say, "Hey we're going to pass on you for this job in Division X but you know you're smart and Division Y's role is a MUCH better fit for you--why don't we hire you for that?" I've never seen this happen in my life.
Try talking to any company's recruiter and say, "You know, I specifically want to work for your company but am not sure what the right role is. Why don't we do an interview and then determine where the best fit is" and watch their brains explode. Nobody does it that way, but wouldn't it be so much better?
Actually that's (almost) exactly how I did it joining Apple. I had already interviewed and been turned down with the group I wanted to join twice. Now I was coming in for another interview with a different hiring manager. I was willing to put up with another possible rejection because I really wanted to work at Apple. So I worked with one of the recruiters and had setup interviews with 6 other groups. I did 2-3 interviews a week and it was grueling but in the end I had several groups that wanted to hire me including the original team I wanted to join. They could only make one offer but nothing stopped me from interviewing until I was blue in the face.
EDIT: Another note, at least three of the roles I interviewed for had homework assignments. None took longer than 2 hours and I actually enjoyed them and the subsequent interviews much better.
>I'd love it if a company were to say, "Hey we're going to pass on you for this job in Division X but you know you're smart and Division Y's role is a MUCH better fit for you--why don't we hire you for that?" I've never seen this happen in my life.
I've had it happen twice (almost 3).
I just find a company, look through the listings, find a job I like then send an email explaining my qualifications and my interest.
I think it works well if the company is small and roles are fairly undefined ("You aren't an exact fit for this, but we want your skills for something else")
I've also seen this happen a few times, and it's happened to a few of my friends as well. Google does it a bit differently and makes you pass their gauntlet of interviews to ensure that you're totally overqualified for any position you end up in. Once you get hired there, they find a spot for you, but many many engineers there are super overqualified for what they end up doing. They end up turning away many qualified engineers because they're Google and have enough applicants anyway. If you're not Google or Apple or Facebook, you probably can't afford to do the same thing.
I've wondered if this might not also be a strategy to drain the labor market of talent that might otherwise work for a competitor or found a competing or disruptive startup. There is a limited amount of good engineering talent.
It's annoying as someone trying to build a team too. Oftentimes I see someone I know I want on my team or in my engineering org but can't Tetris them into the org chart. If your company is small enough and you need more smart folks on your team: HIRE THEM. Obviously there are exceptions to this rule but the right people are more important than parity between candidate and predefined role.
For companies where the number of spots is significantly dwarfed by the number of applicants, probably not - you either increase the levels of indirection involved between "initial contact" and "job offer/not" in order to try to add bin-fitting tiers at the beginning, or you significantly increase the burden of interviews by requiring them to be able to judge for multiple different types of role.
For smaller places, it's the "nobody ever got fired for buying IBM" syndrome - rather than trying to fit based on what the prospective managers/departments want, the initial pass is for ass-covering so that HR can't be accused of negligence if a candidate doesn't work out because they looked perfect on paper. Of course, then these places are left wondering why they have open positions for years...
Small enough companies (e.g. where they don't necessarily have full-time recruiters), I've found that your example at the end actually can fly.
Apple is kind of like that as a side effect. If you have the stamina and have the qualifications for it, you can interview separately for +10 different teams if you wanted to, with independent interview stacks. You won't have a triage interview like you specified, but you could kind of do it yourself.
Homework assignments suck. I was once asked to basically create a clone of Yelp in a week (during which I was working full time). I would rather take a full day off and go interview on whiteboards - come at me, algorithms. What I do not have time for, is setting up an entirely new project and work on it in my space time to impress one dude that will be useless in a few days.
Edit: Why are people scared of whiteboards interviews so much? Don't you use them every day to map thoughts out, working with your colleagues etc etc? I'm not saying some questions are (used to be) absurd - the Google/MSFT manhole thing. But why is it so scary to talk about a linked list, a graph traversal algorithm etc? Most of this is stuff that's taught in basic CS classes.
Sorry to sound condescending but it almost feels like that as technology spreads and more and more people come into the field who haven't actually studied proper CS, they're blaming the system instead of trying to learn.
One forgets most of that stuff over years in the industry. They are not difficult to remember either or solve in real time - but it does require practice for achieving consistency. Only having to revise them for an interview is really a waste of time. Let me elaborate with an real life example - programming challenges that require graph traversal and other approaches. If I give one after preparing and revising for few hours over 1-2 days, I can solve all 3-4 questions in an hour. If I revise for one or two hours, I am likely to solve 3, but maybe falter on the 4th. I am the same person in both the cases. Maybe that elaborates the problem and disgruntlement - it pisses me when I pass some and get rejected in another. There is no consistency in the test. :)
> I was once asked to basically create a clone of Yelp in a week
That's just unfair, inconsiderate and unsustainable, and is also a red flag (ie: I would not go to that company).
Giving up a week of your life for an interview ?
and for free ?
And most importantly -
what if ALL companies did this ? you would have to spend months working from home, for free, just to get to the next level of the hiring process .
I don't know what the solution IS, but I know for sure that it IS NOT a 1-week work-from-home-for-free task .
(edit:formatting and some grammar)
Edit2: people will actually pay good money for a yelp (or craigslist/etc/etc) clone on places like upwork.
And they asked you to do it for free .
Whiteboards are bad for writing programs. Think about how often you write programs on whiteboard, it's mostly for diagrams and system flow not to write for and while loops for hard problems. My thinking pattern is different when I write program I tend to write the skeleton first and then fill it out. Writing on whiteboard although I know the solution I'm just spending time writing which sucks (may be because the only thing I write is a cheque once a month and I'm not used to writing for years). Now if it were typing, within the same span of time I can write two or three programs find mistakes and solve everything. Hence I find coding on computer much better than whiteboard and these days I refuse interviewing on whiteboards and say so to anyone who wants to hire me.
Edit: It's ironic how the so called cutting-edge tech companies can't afford a keyboard, mouse, laptop and a projector which would make things way better. I even had a case where the interviewer changed the question with 5 min. to spare , I explained the solution he said it would work and then told me to write it. I was just erasing and pointing arrows because I did not have space etc. So easy to forget that while typing you just hit enter and get a new line. Draw on white board, write on computer; that's how it should be.
I think there's an inherent collective bias against making interviewing and hiring easier because that would make the hiring market more liquid, meaning their best people will be able to more easily find jobs elsewhere.
Speaking as a business owner actively hiring... the cost of a mishire is huge. It is difficult to understate mishire costs. This means you have to simply spend a lot of time with candidates. Even then you have a narrow window of time, relative to how long the business and employee have to live with one another, to figure out if it is a match. This means you will say no to a lot of people after investing significant time and effort into them (and them into your business). We owe it to our teams to hire well. If that means making it harder, so be it. Think about if it is easier for the wrong people to slip in under the radar. It can wreck culture and team quick. Most teams and businesses dont spend enough time. In Work Rules (Bock) they found the first four long interviews they did helped a lot at Google. After that interviews added one percent accuracy in judging a hire. It is a hard problem and I think the first and foremost goal in any hiring process that is well developed aims to make good hires the #1 priority.
I won't discount your theory of a bias, but I think it is a small influencer if it exists.
Absolutely agree and I'll add that you're shopping in a market for lemons. There are multiple adverse selection criteria against the pool of job seekers. (Excellent engineers are more likely to be retained/less likely to be fired. Excellent engineers are more likely to receive an offer when they do interview. Poor engineers are less likely to be retained, more likely to be fired, less likely to be hired.)
In steady state, the applicant pool is of lower overall average quality than the pool of employed engineers.
There was (who I feel was) a mishire at my job. I haven't seen him produce a single work-product since he started 6 months ago. I get the sense that he's worth more to the company (a large tech conglomerate) as a write-off than he is a producer to the team.
We recently hired an ops guy into a dev role and the guy just does not know what he's doing. Merging his own pull requests before code review or before tests have passed the build system. Downs development/staging servers because he's "testing something" and forgets to bring them back up.
Our place has a 12-month contract before being made perm. Once you are perm you are not getting let go unless it's part of larger cutbacks (government). He was made perm before 12 months and I was astounded. They didn't even ask me what I thought before they did it.
In my experience, hiring is the most political act a company does, meaning individual goals very often take priority over company goals.
Well above avoiding mishires, loyalty concerns are paramount in every organization I've worked for. You'd think that they'd treat hiring as the quintessential bet-the-company decision, and operate accordingly, but this simply doesn't happen. Leadership creates policies and procedures designed to introduce opacity in the process with the ostensible goal of avoiding mishires, but the real goal of preserving power.
Agree. The best thing a business can do is standardize their hiring practices to weed out intentional and unintentional bias and politics. It is hard, though, but possible. Guess that is why I love working in a small business. We keep it transparent and real. We just want to hack things and work with good people.
It's virtually impossible for most companies to manage someone out in less than six months. They're earning full freight salary for each of those six months. Many of your overhead costs like e.g. recruiting fees, new machinery, training, come due within that period and are not refundable.
This only costs actual cash outlay, not opportunity cost of other team members (could be coding but are instead spinning up the new engineer), overhead from management salaries, cost to redo work of an other-than-productive employee, etc.
At Valley salaries hiring someone for a day costs you $100k++. In more moderately compensated locales, lower bound it at $40k or so.
"Managing someone out" is a large company concept. Once they realize the mistake, the owner of a small company can get rid of somebody immediately. It can take surprisingly long to realize there's been a hiring mistake, though.
>>At Valley salaries hiring someone for a day costs you $100k++. In more moderately compensated locales, lower bound it at $40k or so.
Sorry, WHAT?
>>pportunity cost of other team members (could be coding but are instead spinning up the new engineer)
Hey I've got news for you. Developers don't just waltz in and start coding from day 1, they need to be brought up to speed on the systems. Your business-nosed attitude needs a reality check. Developers are human capital, not robots.
> Your business-nosed attitude needs a reality check. Developers are human capital, not robots.
You're being unnecessarily defensive. Just because people are people does not change the fact that it costs companies money to onboard them. This whole subthread is "why is it expensive to hire people?" and one part of the answer is "because you have to pay them money while you train them, and pay their colleagues while they do it". It's not a reflection of any unrealistic expectations--it's the opposite.
> Sorry, WHAT?
Computer, desk, office space: $5k
Job ads: $500-3k per hire is not unrealistic
Time spent sifting through applications: Easily 5 hours per week. If the posting is up for a month, that's 20 hours. That translates to about $1.5k in fully loaded costs for an employee with a $100k salary.
Time spent conducting phone screens: Assume 50 phones screens for one position. 30 min per person (20 minute conversation, 10 minute notes/break). 25 hours. Approximately $1.8k for the same $100k employee.
Further interview rounds: Assume about 10 candidates per position. Assume at least 4 person-hours per candidate, plus at least 8 hours devising evaluations (programming tests or what-not). 48 hours = ~$3.6k. Add in $1k of travel costs per candidate = $10k.
Draw up an employment offer. Assume you've got a stock one, but the new hire would like to make an amendment. Run it by a lawyer. $500.
We're at about $25k. It's not $40k, but it's not free, either. It would be easy to spend more time on any of those steps. If you use a recruiter, it will be higher. If you calculate based on SV salaries and costs, it will be higher.
If that's the figure, then maybe bad hires really aren't that bad. Presumably the company won't have gotten zero or negative value from a bad hire, and even if it was $0, $25k isn't going to make or break any but the scrappiest bootstrapper.
The grandparent post was pointing out that once you've decided to hire someone for one day they'll probably be around for at least 6 months. That's where the majority of the cost comes from.
1. the project slips while you expect your new hire to work, and they don't deliver.
2. opportunity cost of actually hiring someone good - by the time you've realized you've made a mistake, a good person is off the market.
3. you actually have to pay a bad hire for their time. you can't just not pay them. that's cold hard cash out the door.
4. everyone on the team starts wondering, how the hell did this guy make it through this process? are we being run by idiots?
5. if the hire is remote in another state, you have to register your business with their tax authorities, and deal with that whole payroll rigamarole.
6. also health insurance bullshit.
7. setting up accounts, changing passwords/keys when they get canned.
8. actually firing them is not fun unless you are a total sociopath, or they are actively causing damage by destroying value (which is extremely rare -- most people will just silently not do a fucking thing for weeks on end).
9. worrying about a lawsuit because ironically, people who cause damage like to threaten these things.
this is on top of your already full workload of doing actual, productive things for actual paying customers and good employees.
We are a small information security consulting boutique. We are interviewing and hiring more than one person. For a small team making good hires is a big deal (most teams are small, even if the organization is large).
I don't know how else to do it, especially when it is really rare for someone to have the skills we need out of the gate. That means testing (we have a testing process) and time talking with candidates. On the other side, streamlining things, we have made hiring missteps in the past, which had measurable and high cost to the business. For us at least, it isn't hot air. Hiring is a hard problem. We spent a lot of time trying to optimize it to find an ideal way that costs less time.
Consider this: every hour a consultant is interviewing they arent doing research or billable work. So we believe strongly enough in how we do things to put our money where our practices are. Maybe the economics for the Googles of the world are different, but for us it is a real decision to be made.
The cost of a mishire is huge and obvious, but don't discount the cost of passing up on a good hire, which is much less visible. But every time a good candidate is passed on, the odds of a mishire go up. There isn't any way to determine with certainty whether a hire will work out or not before they take the job.
Also, take care of the good hires you already have. Everybody takes their people for granted.
I don't think I've ever heard a manager even say the phrase 'turnover' until they have a few ex-employees that they really couldn't afford to have lost, and a bunch of disgruntled people still there. Once the horse has left it's too late to close the barn.
Individual Germans may not have thought of themselves as particularly bigoted against Jews, but that didn't stop them from participating in regimes that marginalized them.
Most of the people fighting against slavery owned slaves. Jim Crow persisted well into the sixties and it's probable that many of the people against slavery would have been for Jim Crow.
It's very possible for the hiring market to be biased against liquidity even when everyone wishes it were easier to hire.
The market is made up of individuals acting in their own self interest. Why would any individual not make hiring easier if they thought they could gain from it?
Note that Jim Crow/Davis Bacon/etc was a legal regime, a "must discriminate" set of laws to prevent people from acting in their own self interest. What such mechanism exists today preventing me from acting in my own interest?
I replied to you elsewhere but I did just remember that Google / Apple poaching deal. That isn't even a bias, but outright manipulation at a massive scale that hurt the market more than any bias probably could. I am just wondering where in the hierarchy things like this go vs. subtler market forces you are theorizing.
Well, things evolve over time. Market manipulation would be slavery, while bias is just Jim Crow. The whole world is operating under a biased, ugly regime, it just gets better little by little over time.
It's only going to get worse. As automation continues, jobs are going to be less about "being efficient and useful to society" and more about "proving other humans that you deserve to get money". And interviews are one manifestation of the latter.
I think SWE as a profession is in a little bit of denial, because we honestly (want to) believe we are being efficient. But the reality probably is that most SW companies don't do very useful work, but rather, struggling to make ends meet, are fighting tooth and nail to prove (often through deception and introducing complexity) that somebody needs their software. And this of course in turn affects the workers in these companies.
So SWEs will have to admit (mostly like everybody else) that it is actually a game of musical chairs, and if they don't want this game to happen in this fashion, they will have to institute some other way to play this game (of proving that you are useful), for example, by having a profession association and a profession exam.
Think about it. If there would really be that much work to be done in SWE, the companies would just take everybody. But they don't, they are more and more picky. There have been other articles linked from HN (such as http://spectrum.ieee.org/at-work/education/the-stem-crisis-i...) that say the same thing.
I don't think it's reasonable to suggest they'd just take everyone if the need is so great. Culture matters and if you get the wrong type of personalities in you could jeopardize your team and end up losing the, for lack of a better term, better bunch of the group.
I think it boils down to money, which are a manifestation of demand (and thus existence of jobs). If you have money to spent, you can afford to take risks (take people in and then throw them away if they turn out to be bad), you can afford to train people on the job, you can afford to organize the work better (as a canonical example, look at Facebook Haxl - by building domain-specific languages that are easy to use to non-CompSci people, which they are then interpreted on a platform written by a comparatively small group of CompSci people).
Here in Prague, situation is a little different to US (from my reading). SWEs are locally in quite high demand, as we are on the other side of the globalization. So companies often take almost anybody (for a reasonably paid job).
On the other hand, if the demand is stagnant, you will try to get the best people by being very picky in attempt to lower the costs.
I have seen E1, E2, SE and PE used as abbreviations for level of experience (Graduate Level 1, Graduate Level 2, Senior, Principal). These levels might have been applied to a RE (Reliability Engineer), SE (Safety Engineer), EE (Electrical Engineer) etc
So there is already potential confusion in the abbreviations used for various engineering roles. SWE provides some disambiguation. Interestingly HWE seems to be used for Hardware Engineer - when they are generally Electrical Engineers.
I like Triplebytes [1] focused approach to hiring engineers. I think they used to interview candidates to assess their skills rather than depend on whiteboards or resumes. If the candidate passes the interview, they would be connected to relevant companies, take up interviews with the companies and negotiate the offer(s). Now it looks like they changed the process slightly to introduce online coding tests to make it easier to screen more candidates. Edit : As pointed out below, the online coding test is the first step followed by one on one interviews.
He got 14 interviews from 54 companies. As a candidate, that's better than what you can expect from Triplebyte, which apparently only interviews 1/6th of applicants (2000 interviewed out of the 12000 who have applied).
If you don't make it to the interview phase with Triplebyte, then the process might as well be as opaque to you as any company. You aren't given much information about how well you tested and the reason you weren't selected to interview.
It would be great if there were a Triplebyte-like service for those of us in the lower 10,000, but until then it's back to applying to dozens of companies.
14 interviews from 54 companies is downright amazing. Better than I've ever seen. I've probably applied to hundreds of companies throughout my life/career and gained interviews at less than 20. Next time I'm looking for a job, 54 companies is going to be the minimum to apply to.
We suffer from adverse selection bias (as does any online job application). Many companies don't even hire from their online applications for this reason (instead using referrals). The truth is that almost all of the people we don't interview would not have passed (we have a small control group that we always interview, to measure our false negative rate). We do certainly make mistakes (our false negative rate is not 0). But I am proud of our process. It's more open than any other interview I know of. We routinely find awesome self-taught programmers who no other company would interview.
I think Triplebyte's initial technical screen is kind of lame. It focuses on solving problems that aren't all that interesting to me, and they start a countdown timer before I've even had a chance to read the problem. It feels like I'm competing against a microwave cooking someone else's dinner.
I'm sorry you did not like our process! For what it's worth, we've tested a bunch of types of questions, and relatively easy questions answered quickly provide the highest accuracy / time ratio. We also cover a bunch of topics (to give everyone a chance to show their strength), so we don't expect everyone to do well on all the questions.
Yeah, I can understand that. I think you'll probably discover in the long run that you eliminate a whole bunch of people who might otherwise be good hires. It's kind of like only doing analysis on how your current customers like your product, while completely ignoring potential customers. And maybe that's for the best. It'd certainly better to build a recruiting tool that serves part of the market well vs. serving none of it well.
Here are a few things that kept personally me from completing the screening process (which I was checking out because I was a recruiter, and now I'm a developer):
- You don't (or didn't) support Elixir, which is what I've primarily been using for the last 12 months, so I spent a few minutes dusting off the "what's the syntax in Ruby" knowlegde
- I'm wasn't emotionally invested into applying, as I don't really know who you'll be connecting me with, and if I want to be challenged, I'll continue working on some personal code (which is what I did)
- Not having my glasses with me, and a bit of the ol' ADHD, made reading just a bit harder, so the countdown timer was basically reminding me the entire time that I couldn't read fast enough
I'm still breaking into a software job and had a homework problem for a company, with a week deadline. Since I worked full time, I spent the majority of my free time working on this problem.
I was happy with my solution, and disappointed when I was rejected.
Rejection is fine, but what upset me more was radio silence when asking for any feedback on my solution. As a young engineer it would have been amazing to get feedback from professionals.
If you've spent 17 years in industry and haven't written a graph traversal algorithm... you probably don't recognize when you're writting a graph traversal algorithm.
I remember somewhat recently there was a new job site for older engineers. Is there one for companies that don't do the bullshit Google style interview? Something like a noalgorithmdatastructurestechincalinterview.com? It would be great if not doing these kinds of interviews became a hiring advantage for companies and a site like that would help greatly.
Random question: Did you just hand-cut the svg code for the graph on the left side of the page? I took a quick look through the source code and didn't see any evidence of a specific tool.
The reason I ask is that that kind of behavior, where you click on the node and it highlights the paths to that node but grays out everything else, is super useful for complicated architecture diagrams where you want to put a ton of stuff in one diagram, but still be able to see where edges go when you have a big spider web of connections. I've been looking for diagramming software that would create these clickable kinds of diagrams for my company, but haven't found anything that exactly fits the bill (Most tools I've found generate static diagrams, and some mind mapping software I've seen doesn't totally fit the bill, either).
For web things, I would recommend the combination of dagre (https://github.com/cpettitt/dagre), which will calculate node and edge positions, and d3.js (https://d3js.org/), which you can use to layout your nodes as an SVG.
He used D3.js. It produces standard HTML SVG elements which is why it looks handcrafted, but you can see the d3.min.js link at the bottom of the body. D3 is very powerful and can probably be used to do what you want but has a very steep learning curve imo.
> In the prior 17 years of professional experience, I have never needed to write a graph traversal algorithm.
I don't get this. I'm in my early 20s and have professionally implemented graph traversal algorithms at least five times. Graphs are extremely powerful data structures, and if you have them, then you can expect to traverse/search them.
Funny story(now, wasn't at the time but been a few years now). Made it to a final interview at one of the "elite" tech companies.
Was interviewing for a web development position, and the second whiteboard question was about designing a hash table. I of course was confused as I had no backend knowledge. Next one was about a graph traversal, and again I was confused.
So, because of that I was scared shitless that I needed to know that stuff and went out/crammed it.
Ended up interviewing other places, got an offer and didn't have to answer it.
However, because of that "elite" company's questions I went and learned more about graphs, queues, stacks etc.
So while I was pissed off at them for awhile for wasting my time, in the end without them I might not have spent the time learning and progressing through Comp Sci topics.
Unless you're designing to spec, how do you not need to understand graphs and hash tables for front end work? I'd imagine understanding hash tables would be useful in reasoning about caching, and graphs for dependency management if nothing else.
As a front end developer, if you're consuming APIs and building out interfaces, and passing data back to the APIs you don't have to know much about graphs or hash tables.
For most web developer/front-end positions, you can be perfectly fine without knowing much of the "advanced" topics of computer science. You're going to be designing to the product manager's specs, paired with a back-end programmer who is going to tell you the structure of the data that will be passed back down and then you build it from there.
As far as understanding hash tables or graphs, you can really get away with knowing them.
However, I agree with you after learning these topics. There's incredibly more efficient ways to build front end pages with the concepts of a hash table/lookup. Most recently I designed a page that basically uses the URL slugs as keys in a complex dictionary to grab content from a SQL database to render the pages.
In addition to that, it's also just wonderful to be part of the conversation when you go to design APIs or architecture so you can make sure to relay/point out concerns that may cause a lot of unneeded pressure on the front end.
Tl;dr You can be a web dev or front end without much knowledge of hash tables, graphs but I can't imagine you advancing your career very far without that knowledge.
Of course. Web applications are bigger and bigger, and quite often you have some hierarchical data structure in JSON (usually in form of tree, but tree is kind of graph anyway).
You have to deal with these kind of structures anyway. You have to be able to render such data as hierarchy of components, you have to be able to get some information from it, to transform it, to update it.
Tree traversal is almost as common in front-end world as using .map, .filter or .reduce, I think (and using `reduce` also can seem hard on beginning, but it's quite useful).
It is also not a trivial thing to do efficiently though. I would much rather use a mature library to represent and traverse graphs than to try and implement a graph traversal algorithm myself. I do have some knowledge on the subject, which allows me to work with the available libraries, but some things are just too complex to implement from scratch in a timely fashion, I believe.
I agree that sometimes the best solution is to use existing library, but I can't agree that traversing graphs is "too complex to implement from scratch in a timely fashion".
Programming needs some technical skill anyway. And learning "how to use library X" often takes more time than learning "how to implement algorithm X". Libraries are often bloated, bugged, hard to debug or have terrible API. In these moments basic programming skill set is very useful.
The premise of open source ecosystems such as node is that reusable libs are tested by the whole community. Don't reinvent the wheel. I am not saying you shouldn't learn graphs/CS topics. But you should absolutely utilize the ecosystem.
I think it goes beyond subjective likes/dislikes. I personally like reinventing the wheel, you personally believe in ecosystem, these are just our beliefs.
But I think in the long run everything depends on exact project rather on our wishful thinking.
I had experiences when use of library was much better solution than own baked one, but I also had experiences where none of the libraries I had found were quite fitted for the project and many times I just had to throw library and write something from scratch in simple way.
> reusable libs are tested by the whole community.
reusable libs are often FOR the whole community and not for YOUR project. Ofc sometimes you need exactly what rest of community needs, but sometimes not.
The problem I have with giving homework assignments, at least with junior candidates in India, is that very often they just cheat. In one case, the candidate had his college instructor help him answer the questions over the phone, and do the homework assignment. When we brought the candidate in, it took about all of 30sec to realize that the person who had solved the problems over the phone and the person attending the interview were not the same.
I also went through something similar over the past month looking for work. My main problem was that a lot of startups tried to lure me into their teams by offering a really senior technical management role, then once I had passed all the technical tests, the role ended up being 'senior engineer' (or similar) even though they said they were extremely happy with my test results and interviews.
It's clear to me that they had never intended to offer the senior position and just used it as bait.
Other times, they said I would be interviewing with the CEO but when I got there, they put me through some standard recruitment process and I didn't even see the CEO.
This destroys the startup's credibility in my eyes. I cannot work for liars. Making me do all these tests before telling me what I was really applying for shows that they completely disregarded the value of my time.
So this is evidently the wrong forum in which to ask, but what's it like interviewing in other fields? Do chemists have to perform a titration in front of the panel? Do surgeons have to demonstrate an anatomy lesson on a cadaver? Do bricklayers have to build a wall in under an hour?
This article does a decent job at capturing the grind that is interviewing in software engineering. The expectations during the interview process are high and only get higher the longer you've been in the industry. I wonder how much of this contributes to burnout?
I don't think the interview process contributes to burnout. That happens after you get the job. But, it does make recovering from a burnout a pain in the ass. You need to catch up with the latest trends and refresh on the basics like algos and DS. On top of all of that studying you need to come up with something to fill that 2-3 year gap in your resume.
At my company they do a technical test for an hour followed by a technical interview. It's like an exam. The test is decisive.
I prefer this kind of tests, as they only require an hour of effort, and they are objective. All the time a technical test was involved I received a quick and good offer. When I had to convince them about my skills just by talking I either got no offer or I got an offer that was way to low for my experience (like half or worse).
As an example, the hardest problem in the test was a C function that receives two parameters, one by value, another by reference, and it did some modifications to those parameters. The problem was further complicated by having two global variables with the exact same names as the parameters that were used to call the function but with the variables switched (global a went to parameter b, global b went in parameter a). At the end they asked what's the value of global a, b?
20,000 emails, 1000 phone calls, 100's of presentations to get your product off the ground. It never stops.
*Disclaimer: All numbers are estimates. Product can be something you work on internally for a big company to battle office politics or externally when doing your own thing.
Convincing people that what you do is righteous takes time.
I think there will always be mistakes in hiring. There is no way to tell perfectly from a set of interactions however much one can wish.
We are unpredictable, some are great at deception, we have massive mood swings and phases where we are not at our best, we can deteriorate rapidly and get distracted. What is supposed to be a stable reflection is our education degrees and experience and ideally coupled with a few conversations this should be enough.
The problem here is companies have convinced themselves they can find the perfect candidate for every hire and pursue this unicorn relentlessly. Even with all the ado they will frequently get it wrong. It would be more mature and productive to loosen up and deal with the risks than subject everyone to the kind of process.
I've interviewed with one of the "big 4" before, their interview process was very structured and very gruelling. Didn't get the job, feedback was that I wouldn't even meet their junior developer criteria.
I'm not bitter about it, just crossed it off as a learning experience and moved on, but every time I see one of these articles about interviewing for tech roles, I'm constantly wondering what the best process is.
The big 4 tend to hire the best they can get, and have enough applications to pick from a wide talent pool so can afford to spend time doing whiteboard interviews etc, but for other organisations? I'm not so sure.
A few years ago I decided to leave my job, I was secure but unhappy with the company, so I took some time to try to find a good fit. Looking for a new job wasn't a full-time affair, but I'd say I spent about 10 hours a week on it.
I didn't collect metrics like this, but it took about a year and something north of 170 applications/reachouts. I received zero direct rejections. Probably actually talked to about 15 different companies. Phone interviewed with 3 or 4 and interviewed directly with 2.
I ended up not getting any of those positions.
But I don't feel bad about it, the positions where I made it to the interview rounds weren't even the ones I applied to within those companies. Both of the interviews were friendly, but almost entirely pointless and were non-representative of what I would have likely been doing there anyway.
Finally, after really wanting to leave, I just did the time honored thing of asking a few friends who worked at places I was interested in if they'd refer me. Two places said they'd like to talk, and I work at one of them now. This process took about 3 weeks and I ended up in as near a perfect fit as you can get.
Side-effect, after working at this new place for a couple years, those friends I asked, keep popping up every once in a while to see if I'd be interested in something where they currently work...so they've kept in mind that I both:
- switch jobs
- thought of them when I was looking to do it
I'm not a developer, I'm much more into Technical Project/Program Management, so YMMV, but I also noticed that the hiring processes for people like me have almost no steam behind them. Most companies tried real hard to fit me into developer or engineer pipelines, they had almost no ability to entertain anybody outside of that kind of hiring experience. Of the few that I phone interviewed with, at some point the subject of doing a coding assignment came up and I simply told them that we'd all be sad at events around that idea:
I'm a passable coder, but years out of date. I haven't coded professionally for probably getting onto 15 years now. The job I wanted didn't involve any software development, so the exercise wasn't very useful. If I passed it, it both said something about their engineering maturity (and I wouldn't want to work there) and again wouldn't be representative of my work.
The actual hiring machine at most companies seems impossibly broken. I ran into a few good recruiters, but most were just trying to make quota or show effort.
The last time I got a referral from an old co-worker, it got me past the resume screen and no farther. The rest of the interview was identical to what you'd face if you had applied cold off the street. YMMV.
Yeah, I had to do an interview anyways, but the process at that point (for me at least) was more of a formality than a decider. They were basically checking to make sure I wasn't a crazy person and that the reference bonus they were about to pay my friend was worth anything.
This mirrors the experience I've had at several jobs, so I don't think it's terribly rare.
Where it's been different than this has been when getting referred to your everyday startup -- and usually those processes are so braindead I end up in a queue for a programming exchange that won't matter in the end. And I've interviewed at some of the biggest "startups" that regularly make front-page here -- so I'm not talking about weird edge cases.
The discussion I'm reading is absurd. Why has no one suggested a credential yet? Every other profession has it. It's insanely inefficient for our industry to behave like this. Software development and engineering is mature. We need candidates to build stuff not spend time reviewing cs101 or hours doing coding assignments
Credentialing in software probably has a really short half life. I agree with the comment about how insanely inefficient it is though!
I have this near conspiracy theory about why the interviewing process includes such ridiculous things as balancing red black trees. Like every other regulation/tax, the propagation of crazy ideas about interviewing practices is really self serving for the tech giants. In a large orgy of stupidity, companies which can neither afford the impedance of nor are particularly good at these CS type questions mimic the practices because they are probably trying to show that they have a high bar.
Unfortunately, the multi-billion dollar companies can churn through the candidate selection process and waste more money in the process than many other companies' annual revenue. But I am sure they realize the halo effect of making a great deal of noise about this practice, because this is one of those strategies with very little downside and all kinds of unintended upside.
If I were a recruiter at one of the top 10 companies, I would spend half my time creating fictitious accounts on various social platforms to ask about the CS question du jour at these interviews. It would be the equivalent of click fraud in tech job search.
On a more serious note, I wonder if companies trying to hire these employees ever really do a cost analysis of the interview process itself? At what point does it stop making sense to "just wait till we get that truly exceptional candidate"?
Really? Yet it seems that the "standard" CS101 interview is still pretty much the same. Unless you're talking strictly about credentialing for a specific language or framework which I would agree would be stupid. Also, as with other professions we can frequently update the credential to ensure it stays relevant. The IEEE has already created a software body of knowledge. There are fundamentals every software engineer should know that aren't the latest JavaScript framework.
Software development and engineering is mature? I would disagree strongly. I don't see any evidence that there is a particular established right way to develop software yet. We're still muddling around.
I really worry that if someone tries to establish a software engineering system, then that someone will be unreasonably confident, and probably wedded to a particular method or process. Then we'll all be stuck having to prove we know UML (or the equivalent 2017 fad) inside-out in order to get a job.
Anyway, isn't this what university degrees are supposed to be doing? Maybe we just need a more effective way to shame universities that graduate computer scientists who can't program?
As with other professions, we can always create a "professional degree" that follows the standard 4 year degree. Similar to how medicine and law behave. Or how you basically have to spend the equivalent of a year studying to receive your CFA charter. Now further to your point about software not being mature, I'd say the evidence is clear. Software design patterns and languages are actually rather consistent. Software language paradigms are pretty much set. http://wiki.c2.com/?ThereAreExactlyThreeParadigms
Further, that standard commonly used algos and data structures that make a good design are established and basically a standard part of every modern language. What does seem to be rapidly changing is the framework du jour where for some reason we keep reinventing the wheel thinking that we are doing things better when in actuality we just keep creating competing frameworks that don't necessarily benefit anyone. I'm all for competition but eventually a clear winner is defined in a space and it might be better to contribute to improving that winner than not.
Finally I'd say software is in a better place now than medicine was when it was about the same age. We are much more evidence driven (but still lots of opinions persist). And as with any other scientific discipline (because let's face it that's what software is) it will change with new literature so of course we should have a continuing education component.
The reality is, the only party that doesn't benefit from having a credentialed software practitioner is companies because now they have their pick of the litter and can control prices. But if all of a sudden the barrier to entry to becoming a software engineer is high and only licensed practitioners can allow code to be released into the world, we become an expensive resource.
The interview needs to assess skills that will be used in the job. While I can appreciate the spirit of the homework assignment, our role requires client-facing capabilities. With that comes the necessity of being able to answer questions on-the-spot. Sometimes we have meetings with C-level executives with whom we can only get time once every month or every six weeks, and if something happens to come up in one of those meetings, "I'll get back to you in four hours" is rarely an acceptable answer.
I think what would really help transform the hiring process is to assess skills that would be required on the job. I think that was the spirit of the original submission here, but I just wanted to explicitly state that I don't think homework is the solution to remedying the current state of hiring in all cases insofar as actually assessing the skills that would be required on the job.
>Use homework assignments to assess the candidate’s engineering skills
That could be abused. If the assignment takes longer than, say, an hour - it's unfair to have someone waste their time, especially if every prospective job has one too.
With a homework assignment that is not timed, you let the candidate do her best possible work in her ideal environment, and yes, while a work environment is sometimes chaotic, when she needs to get her work done, she is going to make her environment as perfect as possible.
I'd like to second this comment as someone who was recently laid off, did a bunch of interviews[0] and landed a job I'm enjoying very much. The company I picked had a "homework assignment". It wasn't a real project that had any useful purpose to them, which I've seen suggestions about doing[1], but a simple web based app.
At first, I was a little less enthusiastic about the approach -- I have GitHub repos and other examples of competency out there -- but as I thought about it, this "test app" was in line with what they were looking for and when they described the approach, I liked it. The first thing they did correctly was provide a basic spec sheet. It wasn't overly specific and I think this was by design. It was basically a CRUD web app, with a statement of "feel free to get creative; do as little or as much as you feel it needs". I provided a self-hosted CRUD app that worked on multiple devices, syncing state between them in real-time, documentation, an installer and everything else that was needed to make it a complete product. Most of that wasn't on the spec sheet, but in line with thinking about the project as "something I'd deliver", those were things I'd consider to make it complete and it also gave me an opportunity to demonstrate competency in different frameworks, provide examples of functional/OOP design and other areas that gave them an adequate look at what I could do. The app, itself, was a really minimal web app that took me a couple of hours to write, but after all of the additions, it landed in at a day and a half or so -- not a terrible amount of work for someone out of work and worth it since I landed the job.
Frankly, given the choice between shoulder-surf coding, whiteboard coding[2] and test projects, I'll take the test project. I hate whiteboard interviews and thankfully there were none of those this time around. I've had two whiteboard interviews in the last five years. In one case, I got lucky in that the first algorithm was one I had written into an app I had on a private (personal) BitBucket repo, so I pivoted with "I can marker up your board here or I can show you a good implementation that I did a few weeks ago". We spent the rest of the interview with them asking questions about my choices and that was one of the most enjoyable interviews I've had. I didn't take that job. The other, I didn't get as lucky and actually ended up turning down the job on my way out of the interview room[2].
I agree with the author about having empathy for the job seeker. This empathy, though, works both ways. In my case, I had filled in for a dev team in a pinch -- at the last minute -- to run an interview. They hired the guy on my recommendation and he ended up being a great employee so I was pulled in to most of the interviews for several of the dev teams from that point forward. Never mind that I did development on an ops/security team and didn't work with any of these teams, making whether or not they hired a particular person -- or anyone at all -- of now benefit to anything I was responsible for. As a guy who had an already overbooked week, I didn't enjoy being pulled into these and I really didn't appreciate the massive number of woefully under-qualified candidates or the fact that all of that time was often spent on a great candidate who turned down the job in the end. Because of this, I express thanks at various times during the interview process and specifically mention how grateful I am for the time taken out of their already busy work-life. I was glad for the opportunity to interview at all of the places that chose to interview me so I simply made that known. The cool thing about that is later interviews felt more at-ease with the team as a result[3].
[0] I had quite better luck than the author, but did fewer interviews, and was seeking a position that was right in line with my experience.
[1] I like this idea, too, provided compensation is provided -- after all, if you intend to use the product of the work a candidate performs, they're technically a contractor and I wouldn't do work like this for free.
[2] I actually do pretty well on my toes -- I'm good at rationalizing out loud, and breaking up the tension with a little humor (to the point that I've been given hints at answers when I was stuck). Despite turning down this job, they made a really good offer to me two weeks later that I declined -- the job itself was in a domain I wasn't excited about, but the whiteboarding didn't help.
[3] Felt being the operative word -- these were later interviews which means the team was already more interested in me as a candidate than in the first interview. I also remember the first time a candidate specifically thanked me for the time I spent interviewing him and very likely projected that onto my interviewer. But at the same time, it increased my confidence so if it was all in my head, I'm OK with that. I also feel that it was "the right thing to do".
I wish I could get a job someday. I have degrees and great grades and would love to work for so many places but I find the job application process so soul crushing and inhumane and lonely that I've never gotten through it to a job. I don't really think the process is evil or anything, I just have had a lot of emotional trauma so it does not work for me because I can't handle rejection like that. I just wish there was a company where I could go there and work for free for a few months and if I was able to help out then I can stay. Even if I got rejected then, at least I would see what work was like and how to get better and it wouldn't be cramming a bunch of stuff to get through one of these torture gauntlets. AI makes me so intimidated and depressed and I spend a lot of time worried that I will never be able to get a job because I can't keep up with how fast things are progressing. I really really want a job so I can move out of my parent's basement and I would like to go on a date someday and have someone to love and be loved by. Does anyone know a way for high functioning people with developmental disorders to get to where they can have a job? I admit I don't have a lot of intrinsic motivation for technology because it's ultimately nonsense to have a such a thing. Technology is means of achieving goals, not a goal in itself, so I don't believe in technology for its own sake, I just wish there was something that an employer wanted me to do for them but I can't motivate myself to do something useless. I want a job so I can then be able to afford friends and a family, and wish I could find a job it's just that I'm so depressed about my chances that I'm paralyzed. To me the statistic that matters, the real measure of quality of life, is seen in how fewer and fewer people have close friends and I understand how Facebook, automation, etc. are driving that so I am torn between trying to fight my way to where at least I can be on top of the machine rather than under it and by empathy for the way spirits are being destroyed underneath that machine and the material "wealth" or "stuff" it is producing is not a trade that I can begin to find worthwhile even if I try to fake it so until I make it so. Siiiiiiigh. Are any of you problem solvers? Can you help me at all?
It is as from Perlin's foreword to SICP:
"The computers are never large enough or fast enough. Each breakthrough in hardware technology leads to more massive programming enterprises, new organizational principles, and an enrichment of abstract models. Every reader should ask himself periodically ``Toward what end, toward what end?'' -- but do not ask it too often lest you pass up the fun of programming for the constipation of bittersweet philosophy."
I am stuck in this constipation, because I feel lonely, and I feel the computer is driving us to lonelier and lonelier ends, and, as much as I feel terrible hatred at my lack of employment, I would rather play with you than work with you if our work really is making the world lonelier and more and more dependent on anti-depressants to make up for how we have somehow missed something precious, perhaps glimpsed by Harry Harlow, in our mad dash to try to replace terrycloth and humble fare with wire mesh and our cornucopia of junk.
>While I do have some ideas of how to improve the process, I do not have a suggested perfect hiring process. Partially because it is impossible to craft a single process that will work for all roles, for all candidates, for all companies. We need to balance being unbiased and impartial with respecting the experience and background of the candidate. It is relatively easy to do in one-off situations but very hard to do at scale.
The thing to do is to remove proxies for competence and intermediary parties. Talk directly to the candidate, about the actual work (don't try to "see how the handle" odd questions).
However, this requires decentralization of decision making which reduces control.
From what I've seen the best hiring process seems to be Automattic's. They do a short interview over chat then give them a project if they pass the interview where they pay them $25/hr and then evaluate at the end. Minimal time wasted for both parties.
The tldr is: "I wanted a lofty title and a very specific role at some future company. It took a long time and hard work. That's not good for some reason, which I won't go into here."
"I found the process to be dehumanizing, stressful, chaotic, inaccurate, opaque..."
Then goes on to talk about pigeon holing, other issues, and some suggestions.
You could criticize his conclusions or suggestions, but I don't see how you would summarize it as "not good for some reason which I won't go into here".
1. The "assignment" took me about 4 hours, which they expected and told me off the bat. 2. It involved mostly relevant code with 1-2 wrinkles that were not odd just not everyday sorts of things.
For the proceeding interview, I was able to sit with 3-4 engineers for 1-2 hours really just diving into details about my experience and their processes, etc.
So if I could offer advice it would be to find a "homework problem" your existing crew could build independently in 4 hours or so. As in you've actually had more than 1 team member attempt the request, and they did it in about 4 hours each.
This was actually an interview process I didn't mind. I have 15-ish years of experience doing development. I can spare half a day for a prospect no problem. It was actually kind of fun.
4 hours seems to be a good "sweet spot". You can ask something sufficiently technical and oriented to the daily work without being a bore. And when the interview comes around, you can make sure it's not some ego-bound contest, more of a conversation.