The rule of thumb I subscribed to: have the interviewer(s) do the interview. Give the candidate 3x time to do it that the in-house people took. Or, framed in reverse, if you want the assignment to take and n minutes or hours, give yourself n/3 to complete it, and however far you get is the bar for the assignment. Interviewees are stressed, maybe outside their domain, and other things. Make something that they can show you their value. The harder and correct thing to do is to find how they can help your org. It is easy to find what they don't know or to play stump the idiot.
My high school physics teacher did the same. He would take the tests he prepared himself to check whether the scope was appropriate for the allocated time. The factor of 3 was spot on for this kind of estimation.
We did this at my company. A developer at the level that that developer was applying for would do a task that took them 2-4 hours. We then gave the interviewee four days to complete the task with a recommended time of 2 days. We also always made sure to include the weekend in the time they had to complete the task as we know during the week time is precious.
We give a test that should take 1-2 hours, and give them an entire 7 days to do it. I feel like you might be being unnecessarily restrictive in your time limits.
What I've seen is that people spend the time they'll spend on it, and that's it. Only 1 or 2 people over the last few years have said they ran out of time, and the quality of their code showed that they weren't a fit for the company anyhow.
I read the grandparent as: The task should take 2-4h and be done within 2-4 days is how I read it.
I’m still not sure why people expect it’s easy to fit this in and it’s the last thing you want to do after a hard day of work.
I’ve just started taking a full day off for programming exercises I do when interviewing. It allows me to give well thought through and tested solutions even if I make some small mistakes or misunderstandings. Crazy I know but necessary usually these days.
This is why I only do these for exceptional offers. Like a studio I wanted to work at and a problem that seemed challenging. Otherwise it's a strict decline.
These days they don't give you pass even if you ace the take home test.
Apparently its got something to do with the whole team getting together and publicly code reviewing your assignment. And if they feel like the code doesn't fit even to the exact size per their liking, its a reject.
I have had a similar experience, where a perfectly done assignment with unit and functional tests, documentation and all the bells and whistles, It took me two full sleepless nights to get it done. But apparently they rejected, for me not using a specific design pattern which one reviewer likes.
It's absurd to the point of disbelief.
Trust me the take home assignment is a sham. People inside get their friends and alumni hired with no hassles. They only give you these tests because they need reasons to reject you.
I'm interviewing with 6 companies simultaneously and like you they all think they're deserving of free labor.
I'd wager your company must have the worst devs, those who have little self respect and zero other prospects.
Do you not understand that you're requesting free labor from an applicant? Did you forget the purpose isn't to have the sickest challenge ever but to evaluate someone's code/ingenuity?
Many people prefer this to an in-person whiteboard challenge. It's lower stress, and doesn't require taking additional time off from your current employer.
A 2-4 hour challenge is less time than I've spent in interviews at some companies, and would gladly have done it prior to an in-person interview given the option. That way, the actual interview time could have been spent talking about things of value, rather than having them stare at me while I white-board out problems around graph data structures and recursion and whatever other trite trivia they can come up with.
To be useful for discriminating between competent candidates, a challenge has to either be hard to complete (pass based on completion) or hard to complete to a high standard of quality (pass based on completion+quality). In other words, it has to be at the limits of what's possible for people you expect to apply for the job.
If you give me a 2 hour challenge with a timer so I can't possibly take more than 2 hours, I'm fine with that - but you won't accomplish the "lower stress" goal.
But if you give me a 2 hour challenge without timing? I know for sure other people will have spent 8 hours making super-polished, gold-plated solutions. Can I afford to fall behind the competition? If not, I should spend 8 hours too.
> To be useful for discriminating between competent candidates,
I helped redesign the code challenge at one of the companies I worked for. We (the people across the different technical disciplines we hired for) put a lot of effort into ensuring that:
* the challenge could be completed in 4 hours
* the challenge resembled the sort of work people would do on the job
* the goal of the challenge was to inform a follow up interview, not punish people
* candidates were explicitly told that they were not being timed, but that we expected the challenge should take them up to 4 hours to give an idea of the upper-bound level of effort expected
The single biggest challenge was balancing "what do we need to know" versus "what would we like to know" versus the amount of time we were asking of our candidates.
We didn't hard-fail anyone unless they were blatantly not a good fit from their submission, and I habitually wrote multiple pages of feedback for them so they at least got something for their time if they didn't get a follow up interview.
Again, the purpose of the challenge was not a screen, so much as it was to help guide the in-person interview and so we had something concrete that we could discuss.
Follow up edit:
Designing code challenges is hard. You have to avoid something so objective it can be copy-pasted from stack overflow, but objective enough that it can be graded free from personal bias (as much as possible anyway). On top of that, you can't test for MVC+CQRS+FP+SOLID+every-other-possible-thing under the sun, because you have to be respectful of candidate's times.
The previous code challenge (the one I replaced) would frequently take people 8-20 hours to complete, and it was actually pretty simple. As you rightly pointed out though, candidates often read far more into the requirements than were actually there, and would often over-achieve in an effort to stand out. Not only would those submissions waste their time, it wasted ours as well as they took longer to develop feedback from.
I think you're missing the point from the previous poster about (perceived) competition.
Unless your challenge literally stops at, "make X appear onscreen" with no regard for quality, testing, etc. Giving unchecked/unverified time restraints isn't fair. It doesn't matter you're giving more time than it should take to complete. If the task can be done in 2 hours, but you give "6" and Candidate A does it in 3, but Candidate B does it in 32 (but tells you 6) you're ranking two totally different submissions. Candidate B might have a super polished submission, while Candidate A has a baseline submission.
The poster you replied to was suggesting that tests should be either 1) not based on quality of submission and simply rely on difficulty so that only a few candidates can complete them or 2) based on quality and difficulty, but with a checked and verified time to keep the playing field level. However those options are both at odds with "low stress."
Again, in our case, we didn't use it as a screen unless the code was an unmitigated disaster or didn't complete the basic requirements.
So, if someone submitted something that could have been done better, see might ask what could be done to improve it in the follow up interview. A good answer would include the technical bits, and a better answer would describe the time recommendation and why a more polished version would take longer or be considered over-architecting.
This purpose- to inform a conversation with a concrete, familiar code base- is better than both take-home screen assignments as well as the in-person whiteboard exercises, which encourage route memorization and don't reflect the nature of the job we were hiring for.
Right. Some companies use it as a screen, we used it to inform the interview.
Two of the last three companies I interviewed at had at least 4 hours of in-person interviews. One of those (for a regular developer position, around 7 years ago) had multiple whiteboard sessions with different people. The problems were trivial to solve if you had access to google or had memorized basic data structures and related algorithms. Instead of taking half an hour or an hour talking about the code I'd written to solve a problem representative of what the actual job entailed, we spent 4x as much time (during business hours) talking about things freshmen learn in CS programs (I'm assuming here, I didn't get a CS degree).
Even if the total amount of time is the same, or even greater, I still prefer the take-home assignment because I can do it on my own time, rather than taking off of work, and - if done correctly - makes for a much interview.
As a candidate, you learn far more about the people you're going to be working with than in a whiteboard session as well.
2-4 hours is anywhere from $100 to $800 worth of free labor depending on how the dev values their time. The GP was bragging about how generous they were in providing multiple days and weekends to finish it.
How did this become acceptable practice for hiring people? Do any other industries besides tech do this?
So teams should just not vet candidates? An in person interview is also going to be 2-4 hours, plus travel time. Should we stop doing those, too? How exactly do you suggest teams screen candidates?
Designers are hired based on their portfolio, why are devs treated differently? I have a plethora of options on my github as I'm sure most here do. Why is it not acceptable to judge a candidate by their already written code?
Lots of people would see this as unfair, as they don't have anything GitHub. Either for preferring to do something else with their precious free time, or perhaps slaving full days in their closed-source jobs.
I think the only fair option is an in-person interview without homework assignments. Treats everyone the same and doesn't require inordinate time commitment.
Studying for a programming interview is also transferrable, whereas maybe pieces of your weird homework assignment will be applicable in another requested project.
If I want to interview at 6 places that just use onsite interviewing it's easy to figure out how to prepare.
They are asking the interviewee to complete the same task as done by an existing staff member, and compare the results, how is that "free labour"?
I'm not sure where I stand on this sort of thing to be honest, my only experience of anything similar was a pre-interview task to implement a simple 10 entry LRU cache, and after getting the job I was told the only reason it was there was to weed out time wasters with next to zero skill being sent by the recruitment company.
Any industry where hiring candidates with bad fit is extremely expensive will have a long interviewing process. For the candidate there is not much difference between spending 2-4 hours at home at a time of their choosing and 2-4 hours in a first of the N rounds of on-site interviews. Except they will have to expense transport and work around their job schedule.
>the purpose isn't to have the sickest challenge ever but to evaluate someone's code/ingenuity?
This rings so true for me re: the last shop I was at. People got off giving "hard" problems and watching dev suffer and then giving high praise for the ones that could take the heat. That place was really toxic.
How many jobs do you apply to when you're searching? More than a few I'm sure.
Maybe 10% of them get back to you with an interview request and that leads to a code challenge. It's highly plausible for a senior Dev to be simultaneously interviewing with multiple companies and have 5-10 pending code challenges. That shit adds up...
In a government role, I was required to obtain permission for any outside activity, with meaningful penalties. Significant engagements like that for an interview would be in scope.
The interview process being long or whatever or requiring on-site visits is reasonable, but pre-work is probably eliminating good candidates, or attracting people who don't pay attention to their contracts.
Actually, I did one of these jumping from one megacorp to another. It took up my entire weekend, but I landed the interview. I then had to take 2 days off to fly out for the interview. Luckily, I got the job.
You either got very lucky or you're far more effective at pre-screening companies than I am. The last time I spent that kind of time on a test was for a megacorp in the top 50 of the fortune 500.
The process was much the same as you described: multiple day test followed by taking two days off to fly out for the interview. Except I didn't get the job and nobody who interviewed me ever looked at the take home test.
During a recent interview process I was given a take home assignment for data engineering. Well, I had to wait two weeks for the team to come up with a take home since they didn't have a data engineering assignment at the time. Eventually they sent it to me with a 2 hour time limit. I read the multi-page problem description which had bulleted requirements at three different places in the document, somewhat at odds with one another. There was a request to code a certain NLP technique from scratch and create a recommendation engine based on similar user history. I read through the sample data which had no notion of individual users or any table like that.
The whole thing seemed like something copied and pasted together by committee, with a strong smell of "PhD CTO" trying to test for intelligence by quizzing on an esoteric corner of their own grad school experience. After about an hour of trying to figure out conceptually how I could begin to implement the things asked, let alone the 6+ hours I thought it would take in practice, I emailed the recruiter back and said "Thanks but no thanks!"
My other recruiter friends later confirmed that they had burned through almost every firm in town with their ridiculous take home exams that no candidate could pass within the allotted timeframe. I think bailing was a good choice.
> You have ‘4–6 hours’ to design a slick product, with a memorable brand and cohesive working method. No one acknowledges the fact that in reality, you’re about to dedicate up to 5 working days on this task, with the potential for them to ghost you straight afterwards.
To make this more fair, why not let the designer work at the company's office, so the interviewer can actually see how much time it took, and the interviewee doesn't waste more time than necessary?
That's what's I thought too. At one of the companies I interviewed at, we had a programming task that could take max 4 hours. The interview was at the company itself.
Less convenient for people who have a different job at the same time though.
They are not selecting for individuals with an ability to do the job, punch out at the right time, and then live their lives.
Corporations love obsessive-compulsive workers who feel thankful to have the job, are willing to sacrifice personal time towards their business goals and would never dream to unionize or otherwise engage in collective bargaining. They want solitary and competitive workaholics, that's the whole point of 'homeworks'.
A.k.a individuals who accept whatever deadline is imposed onto them and work round the clock to meet it instead of telling management they failed to properly assign time and resources to the project.
I won't spend half a day in your building without being paid. At home, I can split the task. Beside, 4 hours task is a bad joke. I interview at a lot of different companies. I have no time for this.
My policy, before finding a place I like and plan to stick around at, is no programming tasks. There's plenty of companies desperate to hire than you can be picky if you know your stuff. I also will walk out if they start up with those 'clever' programming questions that have nothing to do with the job.
You took out the time to go to some office. You just spent an hour talking about your hobbies and attitudes and the company's working culture etc. Then, a programmer asks you to balance a red/black tree. You do.. what? You angrily throw the marker on the ground, cry insult, and run out the building? No, right? So then what?
I see this repeated a lot on HN and I'm not sure I'd have the guts to walk out of an interview. To be honest, I doubt many people do.
In the current market there’s abundance of junior or mid-level developers from body shops applying to senior level positions. If you cannot stand from this crowd and shine by balancing the tree right, why someone would care hiring you? I can tell you plenty of stories about people who had impressive background and great soft skills, but in the end barely can do their job.
> If you cannot stand from this crowd and shine by balancing the tree right
I don't think memorization of an algorithm is what should make a senior level candidate stand out. Senior people need to be able to lead, mentor, self manage (this is a huge one IMO), and understand the bigger picture. "How do we avoid having to balance this tree at all?"
The role of the senior Engineer isn't to tell you how to balance the tree. You, or anyone you instruct to, can simply ask google that or use a library.
Their role is to ask why we need to balance the tree. You very definitely can't ask google that, and that's why you need to hire a senior Engineer.
It’s one of the ways, for sure. But it is really interesting to me how a person without deep knowledge of fundamentals can lead or mentor anyone while being on a non-management position? They are supposed to gain authority from their professionalism, but without theoretical knowledge they won’t be able to justify their decisions, and then what? What exactly they can teach anyone?
As a senior/lead developer with years of broad experience you don't need to remember details of every algorithm in existence. But you should keep information scrap in your head to guide you or your apprentice to relevant source of information.
"Hi <dev name>! I think we may need to optimize memory consumption of this distinct element counting code a bit; I remember there is some HyperLogLog algorithm which should help here... let's check how it goes... (reading Wikipedia page)[0]"
On a first date you don't do boring everyday stuff becouse you don't want to spend time on that.
My experience is that hireing descicion is so noisy that there are no reason to invest too much since the rate of sucess is barely increased anyway.
The more picky the employer seems the less reason to apply. It might be a internal candidate allready chosen or they are just hoarding resumés to keep recruiters occipied.
The decision is allready made from the from the first minutes ... but you can ruin it of course.
The last time I applied to a company, I ask them all of this. Something in those lines "Hello, I use your product FooBarTools everyday and it help me a lot. I really like it. It is nice piece of software. bla bla bla. I see that you have some opening positions in your careers page. bla bla bla. I would love to be a part of the improvement of FooBarTools. bla bla. What is your hiring process? What do you expect a senior/midlevel/junior should be able to do? bla bla. Thank you for your time. Have a nice day."
> I don't think I'd have the guts to walk out of an interview either, especially for a job I applied to first.
Have you no common courtesy? I terminate the process as soon as I find out I'm not a good fit for the company or vice versa, no matter who initiated first. Continuing with an interview when I'm sure I'm going to decline is just wasting everybody's time.
Not really in this case. If you go to the job you like saying that the shitjob offered you 2x the salary... and they call your bluff? Now what? Take the shit job you are not going to take? For that you could have bluffed with an empty hand anyway.
I was condoning lying about offers or salaries on offer, just that having more offers signals a perspective employer that you're a desirable candidate.
I've heard of companies asking for proof of other offers so saying so and so offered me 2x is a very risky play if untrue.
I just use recruiters to do all the leg work and tell them my requirements. Always had made sure to still bring it up during the phone screening. Only once had they brought me in and started with the quiz nonsense.
I have walked out of interviews as well. Sometimes the interviewer and I discover together that the recruiter we worked with didn't know their job, at all!
More often, though, it sort of was clear from the start of the interview that the company would not be a good fit for me. I would tell them so, and my reasons why I thought so, and leave after thanking them for their time.
If they ask me to do some simple programming task, I ask them if they have looked at my Github repository and what they think they will learn during the simple task they could not from my repository. If they do would have a good answer for that, for example they want to get to know my thought processes while coding, I am happy to oblige. But if they clearly have not looked at my Github repositories, nor have any inclination to customize the interview process, I tell that I do not believe in the validity of the process.
I have also walked away twice during the one-month trial period. Once the company was unable to find a project for me, even though they kept on promising there would be one for me soon. The other time I discovered I was unable to combine working full-time with finishing writing my PhD thesis.
Anyway, I also found that in half of the cases I walked out,
the other party did understand and accepted why.
Of course, it is easy to walk away when you know you will find a job soon anyway and even if you would not you will get some support from the government anyway.
I did. I walked into the hiring manager's office and he barely looked up while asking me to decode the complex declaration on the white board that wrapped twice. My departing words were along the lines of: if you and your team write code like that, this is not a match. So I left.
A cute thing to do whenever someone sandbags you with trivia is to suggest going over to the developer desks and asking everyone in turn if they know the answer. If you find someone that does, you answered the question. If you don't, then you suggest doing a web search, and you have also shamed your interviewer.
If they don't wave you off at the suggestion, you also get to meet everyone and see their work environment.
I remember an interview at Coinbase I should have walked out of. The first question was "what is the angle between the hour and minute hands of an analog clock at 3:15?" It went downhill from there.
I think you're totally right to walk out on puzzle questions. But if you avoid companies that make sure you can program, aren't you worried about the skills of your colleagues?
There's other ways to evaluate if someone can program. A "4 hour" take home test for example isn't a good one as they could easily cheat or spend more time on it. In person tests are often setup to test your memorization which let's be real, how many of us don't google stuff on the daily?
What I feel is more important is a persons thought process on how they approach a problem and come to a solution.
I can offer different perspective here: what about a task which also has some value for you? I rarely give tasks to candidates and only in cases where otherwise good candidate is not familiar with our technology stack. The task in that case is to build a proof of concept on that stack and requires fast learning and general understanding of software architecture. I always give detailed feedback on completed solutions. Even if the candidate was not selected, they still can take something from it and won’t consider it wasted time.
I would never spend 4 hours on a task after hours for a company I'm interviewing for, that is my time and I value my work life balance. It is my humble opinion that a serious interview should take max 1 hour.
If you don’t see any value in learning and trying something new, I can understand it. At the same time there are plenty of people who see it differently and accept such challenge, willing to commit much more time to this kind of task. Maybe it’s not the kind of the job you would be interested in, but then why your opinion on it would matter?
One hour interview cannot be more than just screening, when split 50-50 to give candidates enough time to ask their questions. It leaves ridiculously small amount of time to the really interesting technical topics. And this discussion is important: CVs are not trustworthy, plenty of people lie or exaggerate the facts — only a live talk can verify the necessary knowledge, experience and skills. It may not be enough, true, but hiring someone after barely speaking to him is unnecessary risk. And hiring someone without necessary practical knowledge and without proving the ability to learn quickly is the same.
It's not a marriage, it's a job, and if you can't screen someone in a hour interview then you're doing it wrong. Have two rounds of interviews, whatever, if a company wants me to learn some skill they need then they can offer training as one of the benefits of the job.
if you can't screen someone in a hour interview then you're doing it wrong
Are you sure you meant to say what you wrote? I'm not aware of any place that has figured out how to properly screen candidates in one hour. If you know how to do it, please share, it would help all of us.
Working for this Ivan guy it's entirely possible. Jokes aside though, I've worked for great companies and each had an hour interview and things were fine, people that didn't fit the culture just left and I can't imagine dedicating 4 hours to one interview because frankly speaking I have better things to do and a company that values a work life balance will respect that and keep their interviews short.
This was actually my thought. Looking at the mockups he submitted, it is clear that this was 30-40 hours of work minimum. And depending how fast/slow they worked it could be much more than that.
If I asked a candidate to spend 4 hours on something and they come back to me with 40 hours of work, it isn't what I asked for. Despite it being potentially amazing, that person ignored the task.
A 4-hour task would probably yield 1-3 mockups. Maybe a launch/splash page, the home leaderboard page, and then a page showing the details of a specific user.
Even these 3 pages would be pushing your 4 hour time limit, but that is really the most I would expect.
So if I was interviewing and comparing candidates and one candidate submits a 40-60 hour project while the other candidates submitted a 4 hour project. I can't really compare the two. I would probably throw out the guy that ignore what I asked them to do by spending 40-60 hours of work on something I only wanted 4 hours of work on.
As a general rule when I'm given a programming task that takes 4 hours to do then - IF everything I will need for the task is already installed on my computer and all dependencies bugs and so forth have already been taken care of in these things - it will take 4 hours.
Maybe the same situation applies in Design somehow? I mean yeah it takes 4 hours to make the product if you happen to have all the graphical elements you will be using already downloaded and organized.
on edit: although I have noticed a disappointment in some places that I have not done a lot of time cleaning up the code after writing it and making everything beautiful and writing a good readme to tell them everything to do and so on and so forth and normally I don't do those things because I spent more than 4 hours because there was stuff that needed fixing and anyway if they wanted all that 4 hours would not be enough.
Right. Someone asked me to make a small Javascript backend API. I spent a day installing a dev environment on my personal machine since I haven't worked not-at-work since I last upgraded. I didn't mind doing it because it was interesting, but taking that time would be a dumb reason to no-hire me over someone else.
Coding is like math homework. When your doing the math homework you know when your done, it's very obvious and anyone can check your work and clearly see it's either right or wrong.
Design is like a book report for English class, it's bottomless and can always be improved given more time, albeit with diminishing returns. Design challenges are evil and bullshit, you can poor hundreds of hours into a design challenge and still iterate further.
Agreed, unless they want a good polished solutions, meaning it has been refactored, has decent comments, is well tested, etc.... then it becomes a lot like design where the sky is the limit.
My wife got her first job in the states because of a take home design challenge. Other companies were wary of her because of her foreign degree and no US experience, and she was able to show off in the design challenge they gave her (she spent more time than estimated, but not incredibly much more than that). In this case, it really worked out for her.
well that is sort of the point of my example - I am done with my math homework if the math works = I am done with my programming assignment if the program does what was asked.
But people complain that I am not done with the programming assignment if I have not made it DRY enough, or put in tests, or done a readme outlining more advanced than run npm install then do npm run start.
Programming assignments may not be infinitely improvable but they can often be improved, I just don't consider it worth the effort to improve the work before submitting
Seems like a really obnoxious way to communicate though. If they want somebody to do their best work and really do not care how much time it takes them to get there, they should just be upfront about it: "You have until next Monday, good luck".
If that makes them look like assholes, yeah, I can see why someone would feel that way but at least they are clear about their intentions and not insulting anyone's intelligence in the process.
Are they though? Playing Devil's Advocate, I have a pretty good idea of what I can do in 3-5 hours of solid work. If I was hiring for someone to replace me and asked the same question, I'm fairly sure I could tell whether it had been 3-5 hours or 3-5 days.
It'd be more useful to know they could execute roughly what I'd expect in 3-5 hours. As in they think quickly, come up with something, get the basics down and are ready to progress or not.
...Of course, if they're doing it at home they could spend 3-5 days on something I'd spend 3-5 hours on and I'd never know...
My employer has sometimes given out challenges that take a few hours. We just ask the candidate when they'd like to do the take-home and email it to them on that schedule. Then they have to submit their solution within the allotted time. It's a nice way to make sure the candidate knows it really only will take a couple hours, because neither side wants the candidate to spend 30 hours on a 3 hour take-home.
I don't doubt that it's quite stressful for some people. In general, I think any element of an interview process is going to be stressful for at least some people, just given the stakes involved. That said, at least this guarantees it's not stressful and a big time sink.
Also, I do think that some people are able to complete it in a third the time, but I think the more useful evidence is that many candidates complete it and do a perfectly good job, some even with a healthy chunk of time to spare. If it were a totally unrealistic time goal I'd expect we'd quickly notice when no one finished.
FTA: You have ‘4–6 hours’ to design a slick product, with a memorable brand and cohesive working method.
I don’t think Google necessarily requested a full end to end product. A nice logo/app icon for the “branding” portion and some UX concepts can definitely be fleshed out in a single work day, especially with zero tech/biz requirements IMO.
Maybe you’d need to take an extra day just to brainstorm first... but actually pushing pixels, if this designer compressed his output a bit, could have happened in 6 hours.
> To make this more fair, why not let the designer work at the company's office, so the interviewer can actually see how much time it took, and the interviewee doesn't waste more time than necessary?
I had an interview once where we were on the phone, I got the task, and they told me they'd call me back 2 hours later.
My last company they flew me from another continent to do a 3-days assessment, I worked with the team on a legitimate feature(that would be trashed after as they couldn't use the code because I was unpaid). I liked it, I met the team, I interacted with them, I did the task and got a job offer in the end.
All engineers at the company highly valued the 3 days assessment because we got to select great people to work with, but we were getting too many candidates giving up or false negatives, they eventually cut it down to 1-day despite our protests.
I'm not sure what country you're from or what country you flew to for this ridiculous 'interview' process, but it's one of the many things I loathe about this field...right up there with "rock star" and "ninja" coder.
Honestly it's stuff like this, that has me contemplating giving up this career path because there's just no more room for people in their 50s with 25 years of experience in information technology, who aren't willing to sleep at the office, drink your "insert trendy microbrew here" on tap, and suffer through a ludicrous ask like "we want you to work for three days for absolutely nothing to see if you're a good fit".
Agreed! Who has the time to spend on a 5-day long interview?
If you are traveling internationally, then you have to add 1 day of travel on each side of your trip. Plus the 3-day "interview" or challenge or whatever. So that is 5 straight days of interviewing for a job. That is actually asinine.
If you have a current job (most software developers do, especially the good ones) then that is an entire work week that i have to request off at my current job to go to unpaid work with no guarantee of getting a job afterwards.
I can't believe someone has actually done this. It is crazy!
It's not supposed to be fair. Interview questions are designed to differentiate candidates as quickly and broadly as possible. This is one where the notional goal of completing the project is different than the true goal of seeing how the candidate performs when given an impossible scenario.
The author of this article fucked up when he tried to pass off 5 days of work as an example of what he could accomplish in 5 hours. I am glad he was able to find inspiration from it, but for the success of his startup I hope he also is able to understand other, better ways he could have approached the interview task.
Meta: I assume you're doing a website update at the moment. I prefer the blue! Though I might still use a white version on the homepage to remove emphasis from the header. Also, on you signup page it would IMO look nicer if you pinned the page footer (eg https://matthewjamestaylor.com/bottom-footer).
Thanks for taking a look! I actually have always had a different style on the landing page from the rest of the app. One was designed to be eye catching and the other functional. The landing page is probably due for a fresh coat of paint, though. I'll give some thought to pinning the footer.
That's fine, and probably works, but the other pages look better finished and the menus jump around, etc., which gives a "in progress" feel to the site IMO.
I love take-home assignments. Especially reasonably scoped ones. They allow me to show off my skills. However, my opinion is that it should be paid labor. Give me a day of work and pay me to do it. This values my time, it actually doesn't add all the much more to the hiring expense (all that engineer interview time, resume review time, etc is expensive!), and it makes sure you are screening out early and often and not wasting 20 people's time doing a bunch of take home work for one dev position.
In an ideal world, the task can be something that the company needed done anyways, further motivating the financial insensitive. Your point about general cost of the hiring process is well taken, but I imagine that companies issuing take-homes are giving applicants the same task, and don’t see a point in paying for the same job to be done.
From a practical perspective, I'm not sure that spending 20 hours+ working on something for free is any more/less likely to make you less productive at work than doing it for pay, but I see where the letter of the law comes in.
Every job I've ever had required me to get advance permission to do paid outside work, so such a rule would eliminate most people who are both ethical and employed.
Someone on my team once asked an iOS engineer to add a button to a codebase during an onsite interview. That is actually a horrible test to complete in 45 minutes if the codebase is large (and particularly hard if it’s not super well maintained because even just familiarizing yourself with the codebase can take a great deal of time).
It’s one of those things that sounds easy but really, really isn’t possible to do in the time allotted. I learned a lot about interviewing people that day.
I can't tell if this is brilliant or sadistic. On the one hand, it really tells you who's smart enough to realize the task can't be done in the allotted time. On the other hand, not being able to do something as simple as adding a button in 45 minutes has to screw with the candidate's mind. That sort of thing would shatter what little confidence I have. I'd probably tank the rest of the interview.
> I learned a lot about interviewing people that day.
Yeah, adding a button to a codebase is an awful test.
If someone made it to an onsite and you don’t know whether they can add a button in swift, something failed in your screening process. If you’re testing how someone navigates a codebase, you can just look at it with them, and let them drive the chat.
If you’re testing an engineer for a serious job, do an algorithms test. If you’re testing an engineer for a specific thing, test that specific thing. Both of those should be handled in the screen, not the onsite.
IMHO, the onsite is about seeing how people think. Whether you can jam a button into a repo doesn’t tell me whether you can think or not. I guess it tells me whether you get flustered, but it’s pretty unfair to design things that are impossible just to see if people break.
That candidate turned out to be awesome but I remember the interviewer telling me after they interviewed them “well, they couldn’t add the button, but as they were doing it, I realized I wouldn’t be able to add the button either so my interview was inconclusive”, and I replied “well it sounds like you need to design a better interview question”. The worst part was, the interviewer spent the first 20 minutes of the interview talking with the candidate before giving them 25 minutes to add the button!!
There’s a question we gave candidates really early on which we no longer do because it biases towards math nerds that I absolutely loved. It’s based on a movie called 13 Tzameti. I’m probably screwing it up because it isn’t my question but it’s basically like this:
You’re in a dark room after being abducted by a gang. The lights come on and there’s 12 other people in the room in a circle and everyone has a revolver with 1 bullet in it (6 slots in the chamber). Your instructions are to spin the chamber on the revolver and, when the lights go out, shoot the person to your right in order.
What are the odds you make it to the next round?
It’s a crazy question and, what’s even crazier is that for some reason, as you increase the number of people in the circle, the odds of making it to the next round converge on 1/e. No one has figured that out in the interview. Also no one has figured out why it converges on 1/e so if you have any ideas, let me know.
I like this question because it shows you how free thinking people are. I dislike this question because it biases towards smartasses and probability nerds.
Your odds go way up if you repeatedly fire at the guy to your left.
Being a senior engineer is knowing the difference between doing what your told and figuring out what needs to be done to achieve project success. In this case, getting to the next round.
He never said you had to spin the chamber by an unknown amount, aren't your odds even better if you make sure the bullet is in the next chamber before you shoot left?
Of course, many game theory calculations assume all players know the payoff matrix and equilibrium strategies of the others; making sure the bullet isn't in the next chamber is the rational universal strategy if the game has a fixed number of rounds.
Presumably the actual question has a bunch of provisos making sure you can't intentionally miss, or accidentally miss, or dodge, or shoot early, or fail to pull the trigger, or shoot the gang.....
The murder question sounds quite interesting, but I'm not sure I understand it. You have n people (initially n=13) in a circle, firing shots that are fatal with probability p (here we apparently fix p=1/6, but I guess that for large n the final answer doesn't depend on p provided it's neither 0 nor 1?), and then 1 fires at 2, then 2 (if alive) or 3 (if not) fires at 3/4, and so on. And you're person 1 (this isn't stated explicitly, though...) so you're the last to have a chance to get shot at, and then you live if person n dies or if person n lives but misses.
So, let's see. Suppose n is very large; then the probability that you live is approximately equal to the probability that your predecessor does; call that q. Then, as above, you live iff your predecessor dies (probability 1-q) or your predecessor lives but fails to kill you (probability q(1-p)). So q = 1-q+q(1-p) = 1-pq and 1 = 1/(1+p).
That's a long way from 1/e, and a quick simulation seems to confirm this answer. It doesn't change a lot if we assume that a random person always fires first, instead of you, or if we assume that you're always last (which I think was the situation in that movie).
If everyone has a revolver with n slots and one bullet, and they all fire at _you_, then you have a 1/e chance of survival for large n, but that sounds too different (and too easily found to be 1/e) to be the right thing.
The probability p(n) that a permutation of n things is a derangement -- which tends to 1/e as n->oo -- satisfies the recurrence relation p(n) = [(n-1)p(n-1)+p(n-2)]/n; is it possible that the correct statement of the problem here leads to that same recurrence?
Ok, I talked to the engineer. I got it mostly right but everyone shoots to the right as soon as the lights go out.
The question is "what's the probability you die?".
Edit: You can also challenge people to think about the problem where everyone fires at exactly the same time OR random order since people have different reaction times.
Edit 2: "The probability p(n) that a permutation of n things is a derangement -- which tends to 1/e as n->oo -- satisfies the recurrence relation p(n) = [(n-1)p(n-1)+p(n-2)]/n; is it possible that the correct statement of the problem here leads to that same recurrence?" <--- My engineer says that derangements are the correct key to the convergence.
I think I still don't understand what's happening in this scenario.
If everyone shoots simultaneously (so in particular everyone does get the chance to shoot) then I die iff the one person shooting at me hits me. Probability equals probability that a given shot hits (so in this case 1/6). No dependence at all on the number of people.
If everyone shoots sequentially, this seems just like what I described above. Probability of death is now p/(1+p) instead of p, at least if you're first to shoot and n is very large. (Unless something's very broken in the heuristic argument I gave. Let's try another. First approximation says a fraction p of people die. But that's not quite right because people who die don't get to shoot, so next approximation says we get p(1-p). Next approximation says we get p(1-p(1-p)). Etc. We can either solve the obvious equation, or else notice that we're getting more and more terms of the binomial expansion of p/(1+p).
I don't see anything here that doesn't look, in a crude approximation, like a fraction p of people dying (p, again, is probability that a given shot hits, which in this case is 1/6).
I must be misunderstanding something in the problem statement here. Perhaps it would be clearer if I'd seen the movie?
Oh, what about this version? You shoot first, things proceed cyclically, and we keep going until just one person is left. What's the chance that it's you? Naively it seems like this should be approximately 1/n no matter what p is; shooting first could confer some advantage but surely it can't be much for large n. So this can't yield anything like 1/e either. Drat.
How about this: People shoot in a random order (now everyone has the same chance of making it to the next round. Is it 1/e?)
import numpy as np
class Shooter:
def __init__(self):
self.dead = False
self.right = None # The person to the right
def simulate(n):
# Simulates a round with n shooters
# Returns the ratio of survivors
shooters = [Shooter() for i in range(n)]
for i, shooter in enumerate(shooters):
shooter.right = shooters[(i+1) % len(shooters)]
np.random.shuffle(shooters)
HIT_PROBABILITY = 1/6
survivors = n
for shooter in shooters:
if not shooter.dead:
if np.random.random() < HIT_PROBABILITY:
shooter.right.dead = True
survivors = survivors - 1
return survivors/n
Simulations suggest that the survival probabilities do not converge to 6/7 if you add shuffling. This makes sense, since the survival probability for the random shuffling version must be strictly less than if you are guaranteed to go first.
>>>sum([simulate(10000) for j in range(1000)])/1000
0.8463075999999998
"The current statement of the problem is that you have n participants, with 6 slots (1 loaded) in their revolver, each firing to the person on the right."
I'll bite, but how do you know it converges to 1/e if no one has figured out why?
Your description sounds close to the wikipedia description of the first round of that movie (https://en.wikipedia.org/wiki/13_Tzameti), so hopefully it's not just the wrong probability to analyze, but naively, the setup sounds like it would have slightly above 5/6 probabilty of survival for the first round. But some aspects of the question setup also almost remind me of the classic networking algorithm of slotted Aloha, which does yield an optimal utilization (packet survival) rate of 1/e in each round (for an altered question setup).
Let's say that the project is a very large, somewhat famous public open source piece of software that has been coded on by many, many different people with different coding styles.
I rather like this idea in the sense that it tells you a lot about your own codebase. There is a lot of stuff out there that only makes sense to the author.
This reminds me of a take home software engineering interview I was once given via email. Same deal, I was told about 5 hours. I’m an iOS developer, so I was expecting a pretty simple app.
I opened the PDF to find not one, but three separate tasks. Completion of all three was expected, with an estimate of about two hours each. One of the tasks was to replicate Apple’s ‘Reminders’ app in its entirety, backend sync functionality included. Another, a task requesting Visual Studio (iOS devs have no need for any experience with this).
I promptly replied declining to continue the interview process. If you’re ever in a similar situation, interviews can sometimes tell you more about the company than they can learn about you. Good chance I dodged a bullet, and could have been working for someone setting highly unrealistic client deadlines, with the expectation that I can build something in any technology proficiently.
> They presented three design challenge options to pick from, with a weeks notice and advised that we should spend no more that 3–5 hours on the task (wink, wink)
I guess as a programmer I'm too logical. You tell me X hours, that's all I'm going to spend. For one of my past interviews I was given a task to make a multiplayer battleship game using whatever I wanted, and was told to spend 2 hours total and they didn't expect me to finish.
I got some rudimentary client/server communication going and that was it. No game logic at all.
Didn't get the job: "However, we would have liked to have seen you get further on the project in the same amount of time."
I think we interviewed for the same job. I did the same thing, and got the same response.
It was such a short timespan that I assumed it was a trick question, and they wanted to see if I could come up with a creative solution that technically met the requirements but was extremely simple. I implemented a board which you could put pins on, and a text-based chat system using websockets. The idea was that it would work like a table-top simulator.
Nope, turns out they wanted a real solution. I wonder how many companies who use these examples actually attempt them before using them to hire people. My guess is they might be more realistic if they did.
Yeah, 2 hrs barely touch anything new in my current job, where we have a lot of boilerplater and in-house infrastructure built by the devprod team. Some companies either take those for granted or just deliberately make the bar extra high for arbitrary reasons.
I hope one day, well I actually hope to be alive to see, that one day humanity will have enough understand of the brain to teach us our cognitive limitations and how to effective work around/out them. Not long ago I quit one of my jobs because I simply couldn’t learn to be effective due to meetings and all the interruptions commonly seen in an open office/agile/start-up company. Seriously, I can’t be a better engineer in such environment.
> you’re about to dedicate up to 5 working days on this task, with the potential for them to ghost you straight afterwards
I often give companies a list of open current open source tasks I'm working on. Pick one and I'll complete it for your interview. Every singe company turned me down, except one, who just accepted one of the tools I had on Github and examined that.
One of the biggest challenges I have as an interviewer is setting up a framework that allows me to compare candidates fairly. Most candidates don't have a list of open source tasks they're working on. If I have three candidates who completed my standard interview code sample and your open source pull request, it's much easier to spot the differences in the other three candidates.
This is a very good idea! I could even imagine something like this could be a socially acceptable way to test candidates, with thw benefit of helping open source
I recently Skype interviewed for a local startup run by a former Googler. He was very proud how much the interview process was based on Google's, with multiple stages to ensure they get the highest quality developers.
The startup sounded interesting, and I might have been prepared to spend the recommended 5 hours on the code test, had I had a chance to actually go into the office and meet the team...!
At least at the end of a Google interview you get to work for Google.
In a data science interview at a respected company I received a take-home with the framing “A Product Manager wants a dashboard with a massive amount of features (including interactivity, model prediction and GIS)” and received only 48 hours to do it. I thought there was some weird trick because that assignment had an unreasonable amount of scope in that timeframe. I found out the company uses a BI tool to streamline such tasks that is only available to enterprises and not consumers.
I eventually made the dashboard but it took 16 man hours; on the on-site, the interviewers implied they didn’t like it as it was not feature complete. (I called them out about the BI tool; they weren’t happy about it but admitted I was correct that it would be more efficient)
Now that I have had more experience as a data scientist, the real-world response to such a framing is to push back against the PM and write an implementation spec with a defined scope.
A previous manager once told me how to deal with product managers: "You tell them what you need from them. Not the other way around." Don't let the PM write the spec. They state the problem and you ask them questions to come to the solution. You also give them the timeline. If they don't like it, then cut features.
This happened to me too. A company reached out to me on Angel with their coding "challenge" - create a facebook timeline clone with an API, enzyme + selenium tests, documentation, and deploy everything to AWS. fortunately i have enough experience to say no to this kind of thing, but I do feel that some more junior folks are being exploited into thinking this sort of project is normal for an interview
>Once one agent learns how to become more competitive by sacrificing a common value, all its competitors must also sacrifice that value or be outcompeted and replaced by the less scrupulous. Therefore, the system is likely to end up with everyone once again equally competitive, but the sacrificed value is gone forever. From a god’s-eye-view, the competitors know they will all be worse off if they defect, but from within the system, given insufficient coordination it’s impossible to avoid.[0]
I once had to do a full-stack coding task for a YC company - I think it was something like build Tic-Tac-Toe with backend validation and storage of game state. I built it, and instead of being called onsite, I had a phone call with an engineer. I expected to discuss my solution with the engineer and prepared accordingly, but they did not mention the assignment at all, and proceeded to ask me algorithm and database trivia questions. I did not get the job.
A while back I made a career change and I was looking to break into the land of programming, where there are a lot of bootcamp noobs like me trying to do the same.
I did an interview and one guy just didn't want to be doing interviews it seemed. Later when I was asking questions it became clear that he was a "senior dev" who just didn't want to talk / work with anyone he deemed less knowledgeable or just not capable or something. I also find out he's the lead for the spot they're hiring... bad feelings started for me.
Later I got some positive feedback that the take home assignment I completed was one of the only fully completed and "thoughtfully done" assignments they received (one of the only times I received useful feedback during a recruiting experience).
Bad vibes aside, I was a noob and beggars can't be choosers so I was surprised when they asked for a second interview and felt I had to go to the interview (need a job!). Second interview and it was the same thing, and when I asked questions he didn't even answer them really / his random technical statements seemed like sort of ultra truisms / not related to the actual problem we were working. It also seemed this dude's team kinda worked on their own island (kinda appealing) and he was the guy running the island evaluating people (very much not appealing). More bad vibes....
It was a big corporate place (good pay, benefits I had heard) so that meant, MORE interviews if we were going to move forward..
But by that time I had a good interview at a small place (less benefits, probabbly less pay, fewer people to learn from / with, long commute, but it seemed friendlier and the lay of the land was way more clear)... I decided that I just had too bad a vibe from the guy who would be my boss so I declined the interview.
I was pretty honest with the HR person that just from the interview this guy really seemed like he didn't want to hire me / didn't really want to work with anyone like me and if they were going to hire someone they might want to work on that. The HR person said "yeah we know".
I still wonder what that job would have been like, would have been really nice to work at that place...but that guy... you just get that sense in an interview sometimes.
We've joked about giving potential hires a challenge to fix a problem we are currently experiencing, but we wouldn't really do it. That takes some balls to ask interviewees to write or fix production code.
Back in 2000, I had an in-person interview at a very small b2b website. They'd been using overseas contractors (zero local technical people) and the founder/interviewer barely knew a thing about computers. He didn't know how to run a technical interview, so he sat me down at a computer, logged in as root on prod (lol the only server), and pointed me at a couple of bugs. I knocked them out lickety-split and was hired on the spot. The world sure has changed
I'm making it like that again. I'm back down with the blue collar proles in Mountain View, landscaping and moving and waiting tables and the like. That's how things work outside of speculation... employer has a business where they get paid, and they outsource their own functions. I'll never run an engineering org like Silicon Valley again, it's so broken in delivering results as a result of the speculative nature. I have a project website up serving two cities. We use Clojure and static generation, and operate and maintain IT like an old tractor. I train my replacements to keep myself available and customers getting a good price (minimum wage). It lets people start their career with hands-on work. It's in BigTech's interest to make us think tech is hard, but I'm using most of the same stuff as I did when I was 14 and self-taught 20 years ago, and it's gotten far far simpler and better.
I'm in high tech and dying. This is making me feel nostalgic and thinking... maybe taking 3 steps back would be 10 steps forward in terms of my mental health.
Every company I see "really would like" to fully automate their build/test/deploy pipeline, to actually have automated tests for most new features, infra as code is a work in progress, et c. Basically most of the Joel-test or modern HN-bubble best practices are still on the wishlist or on a perpetual "we're trying to make that happen" an awful lot of places.
And they are all, all, "fixing their Jira workflows". Constantly. What a time-sink of a tool, as commonly used. I'm convinced at this point they'd be better off letting their dev teams use whatever, and just have human-driven processes for collecting all the (mostly meaningless) metrics they always seem to need to report upward, rather than trying to make one tool do everything automatically. Pretty sure Jira and other heavy PM tools are typically the sort of software/process that Graeber calls out in Bullshit Jobs: adding a ton of work out of proportion with any real benefit, to make everything fit in a computer and on a spreadsheet, rather than reducing work.
It's great if you do it as long as you pay for it. I was doing a 1 week payed interview at Coinbase. I was not hired, but my memories of the interview process are great.
This sort of thing mostly works fine for students and similar. (And internships are often great for everyone concerned.) But this doesn't really work for someone who already is working full-time. Some pay is fine but there's no way I'm taking a week of vacation to interview with you.
At our place we reserve coding tests for applicants on the more junior end of the spectrum. I've always found conversational interviews sufficient for vetting mid to senior level developers.
We do pay a good rate for the time we instruct people to spend on the coding test. We value our people and prefer starting the relationship generously and in good faith.
Also keep in mind that many tech companies have anti-moonlighting clauses or otherwise forbid side projects. You’re losing current employees from these companies from the interview funnel.
I'm sure a lot of folks take the attitude that it only matters if you get caught. However, unless perhaps I already had one foot out the door, I'd be very hesitant to flagrantly violate an employment agreement forbidding such things to accept a relatively modest payment for some outside work.
I understand it, I wasn't a student, but I didn't have a job at that point (I quit 2 years before to travel the world and had some contracting jobs).
The good part is that it was the last step, and 50% of the people from the last step were given an offer, so it wasn't that bad (though I'm afraid that the offer itself was maybe too low compared to my previous works).
I've just completed an interview process (waiting for the decision) that was explicitly "this is a real problem we have", but also stating that I'll be paid for 8 hours work. I have no problem with that. It took about that long as well.
So I suppose if you're honest about it, and you give a fair estimate of the time required, then what's the problem?
I still disagree with that. are you being paid 8 hours at a pro-rated salaried rate? Well you're going to have to report that as income and take a 30% haircut off it. There's a reason contractors get 30% or more above the hourly-equivalent of a salaried employee.
Our interviews have consisted of something similar, but never in a way that produces actual usable work. We've had candidates "add features" to code, except the code in question was in a language they don't know. So they didn't write anything. We have them tackle past PRs for bugs that have already been fixed. And we've had them do design work for systems/processes we've already built.
We're not evaluating them on whether they got the same answer as us. We care about hearing aloud their thought processes.
What's the copyright situation in America anyway? I can't see company lawyers over here be comfortable with using code written outside formal employment.
Code is copy-written by default. So unless the company has you specifically license the code in a way they can use, there's defiantly some room to go after them.
A game studio I worked at had a blanket policy of destroying all fan fiction. And story plots we were sent just because someone had previously sued for stealing their idea.
Means a lot of letters from kids get read only by the secretary till the identify it and destroyed.
To be fair, even though the hourly expectations are often unrealistic, I much prefer a coding challenge (as a former hackathon goer, I likes me a challenge) over a sweaty nervous “whiteboard session” where they nerd grill you on some irrelevant algorithm questions. Unfortunately, most companies do both.
I had a similar experience once but a little more extreme. On a take home interview test they were asking to build a full backend analytics setup plus a data viz UI. I called the recruiter back and told them an estimate to a problem like this was well beyond the ~4 hours they originally told me. Their response was to say "they were looking for people who would find a way to get it done no matter what". I immediately stopped the interview process without looking back.
Had a similar security interview with a healthcare company. Three tabletop exercises that were all dumpster fires (no controls, no logs, no ability to research, etc.)...it was at that time I REALLY looked at the interviewers and noticed just how burnt-out they looked.
You can learn a lot about a company's pain points by the questions they ask during the interview...chances are they're problems they're struggling with at the moment.
I once received one of these tasks, described as requiring four hours. I had a look around the web, found the source of the exercise, complete with an expected time to complete of four hours, and the guy had completed all of two classes before abandoning it.
I've found it incredibly glitchy. I have some reminders that somehow duplicate and show up multiple times (all at the same time), and others that will show up and instantly dismiss themselves. I can even be looking at my phone and see them pop up and instantly disappear.
For a long time I thought it was perhaps due to poor sync with the Mac, since sometimes I manage reminders from there, but the same thing happens with the wife's phone and she doesn't sync Reminders. She has a single daily reminder at 10pm. Sometimes it shows up 4 times. Sometimes 4 times and 1 disappears for a net of 3 reminders. Sometimes just 1 shows up. Sometimes none of them show up. It's bizarre.
Interesting — does iCloud otherwise work normally? We've used it heavily for years and never seen anything like that. They used to have an option in the iCloud.com settings (under advanced) to reset the sync system which might be worth looking for — back in the day there used to be a utility to reset the Mac sync database which often fixed problems like this but I believe that was removed at some point.
I believe iCloud does otherwise work normally. Reminders is the only app I can think of (including all Apple and non-Apple apps) with any sort of obvious glitches, sync-related or otherwise. I'll give the reset a try--thanks!
Reminders sync when I create them and recurring reminders show up on every device, but I have to mark them complete on every single device. Sometimes they show up again right after being marked complete.
I've stopped thinking about it as "glitchy" and more "the new Apple way".
Ridiculously glitchy for me too. No idea if it will remind me or not. No idea if clearing something will really clear it. Unusable really. I assumed it was piss poor exchange/activesync support but if it happens to others without any sort of syncing maybe there's something more to it.
Not Reminders, but Apple's calendar system torpedoed my Apple interview. Didn't properly account for the fact that my interviewer was in China and I wasn't. It got the time of day right, but he called me a day before my interview time while I was in class because it was tomorrow in China.
He let me reschedule for later that day but seemed hostile and grumpy throughout the whole thing. Didn't get the next round.
Recreate Reminders? Wow. That app has an entire team of engineers to build that. It isn’t trivial, despite being “simple.” And to do it in two hours? That’s just insulting.
I was given a take home exam that I took one look at and said nope, I wouldn’t enjoy working for the company. I actually appreciated this approach as a quick way to filter out job opportunities.
>> Another, a task requesting Visual Studio (iOS devs have no need for any experience with this)
Ironically enough I just moved to another iOS job and have been sitting in VS for the better part of the last couple of days. Some shared systems are written in C# so all developers basically have to use it at some points.
I agree it shouldn't be a part of the interview though.
When people are turning their interview questions into successful companies, maybe it is time to start asking some easier interview questions. This is a visceral demonstration of how absolutely ridiculous interviews have gotten.
I think that is crazy. You can turn laundry folding into a successful company. It's not that the interview question bar is high, it's that the "what can you make a company" bar is effectively on the floor. This is good.
Also, OP launched on product hunt today. Calling them a "successful company" is generous if not outright false, though I wish them every success.
Stated less dramatically, what happened here is "I turned my design interview prompt into a real piece of software". That doesn't imply that google's interview questions are too hard. A software company asking for a deliverable that approximates software is sane.
I think the time that google demands for their take-homes is a bit much (explicitly, this is 4-6 hours. in practice, it's 20 to 30 if you want to deliver at high caliber). But the questions/prompts themselves are intentionally quite boring, and exist to see how far you can take it.
I found them to be pretty good. These kinds of questions depend somewhat on chemistry between candidate and prompt, so they ask more than one of them, which is great. The takehome I picked was the only one of the 3 offered I could even pretend to care about. Of the two asked onsite, one I felt I delivered on at only the most basic level, and one I think I could have credibly patented / raised a series A for were it a thing I cared to spend 2 years on.
>explicitly, this is 4-6 hours. in practice, it's 20 to 30 if you want to deliver at high caliber
This is a problem with a lot of take home assignments of many types (not just programming). If someone has landed an interview at a company they really want to work for, it's almost not rational for them to just bang out something that's "good enough for government work" some evening rather than taking the time and care to really do it properly.
But this doesn't scale if they're interviewing at a number of companies and/or otherwise just don't have much free time.
I call them low self esteem corporations, the interview process requires you to constantly flatter them and tell them how much you'd like to work for them. I have too much respect for my own time, what's important is on my resume and the rest they can find out in a short interview, homework assignments and cover letters are plain silly.
Or increase the performance of Gmail (or Google Maps!!) back to what it was. Funny how the two apps ran perfectly smooth on devices 10 years ago, but they struggle with the horsepower on devices today.
The top 1%-2% who are willibg to actually complete the challenge. Probably not even then, as some of the top candidates who do complete the challenge may hold themselves to the explicit time limit.
Personally, I love interview questions with some real life practicality to it. Tell me a problem your company is facing now and I will be energized to give you a solution. Typical abstract questions like design a binary tree or solve an arbitrary puzzle don’t interest me.
And sometimes its easy to memorize such solutions. Translating a real life problem into a computer science one requires some skill.
I'm not a big fan of extended take home assignments but I can definitely see the logic of wanting to have some evidence that a candidate can do what the job requires.
I do a lot more writing than programming and, for someone for which that's one of their primary jobs, I would absolutely want to see writing samples--and, if they didn't have one they could share, they're going to need to write something custom.
I honestly don't get the max 1 hour interview process. You're probably not that special. I guess I did just have an interview over lunch once but I had known (and worked with in various capacities) the person making the hiring decision for years. Other than that, I've always had multiple interviews, if only so that multiple people could talk to me. As an interviewer, I really like having the perspective of multiple people precisely because it is a very imperfect process and people miss things.
I don't understand why interview tests are so prominent, why not just hire the person as a test for a bit and fire him if he's not up to par? Doesn't cost much and is a much better predictor of success in the company
It costs a lot to hire someone and onboard them. And, if you make a practice of firing people after 60 days because they aren't a good fit for whatever reason, you're probably going to find that you have a hard time hiring anyone.
As I said in another comment, (paid) interning and hiring someone as a contractor/consultant for a fixed period are great for situations where it works for everyone. But, if I'm looking for a new full-time position, I'm almost certainly going to pass on a position that doesn't carry with it the presumption of ongoing employment.
[ADDED: And, yes, some companies have an official probationary period but AFAIK that's mostly reserved for when a hire really isn't working out for some reason.]
This startup is the equivalent of a task management app. I guess you can call that a startup, and app to help you manager your exercises. There are only a 1,000 others out there.
Urgh. I recently had a similar situation. I started interviewing with a company who have a take-home assignment which I could have completed in the required time without taking any care over the quality of code I was submitting, but to do it properly I spent a bit of time thinking about it and building a reasonably extensive test-suite along the way.
I was asked to come in for a face to face interview, for which I made clear I was having to take a day off of work, which was cancelled a few working hours before (so I wasn't able to cancel the holiday) without an apology. I was then interviewed again by phone and told that there was 'a bug' in my code and that I should try to find it and resubmit it without any other details. The spec was a puzzle with lots of weird edge cases and horrible inconsistencies.
I decided at that point that I didn't want to work with them.
I had one a little while ago where I spent a fair amount of time on the request, and I thought was pretty well done. It completed all their tasks, showed how to use their api, and it showed I know how to integrate several different technologies, from front things like React to building an API to interact with theirs and setting up and configuring cloud technologies. They came back, and said, "We don't want to continue, we expected more from him." I sat there baffled and so did the recruiter, as it clearly did everything and more than they asked for in the assignment.
I am so glad they didn't want to go further as I am sure they are a nightmare to work for.
There have been anecdotes on HN about companies using this type of process to get free work out of people. If your code solved the problem they needed solved, they may have taken your code/ideas and used it and left you by the wayside. I hope that's not the case but it's possible.
A few things ... the word 'Crunchr' is real hard to make out when overlayed on the image of the person mid-pushup ... in that same-but-too-small font is your company name 'Tona' or 'Tonu'? ... the busy graphics on almost every screen looks great at first but would wear me out and get between me and my info, and looks too much like a cult-of-body-and-youth or like Men's Fitness, definitely not relatable ... the need to select a fancy background for every class listing would be tedious and would require extra info on whatever back end manages schedules, nobody will go through all that work and will be left with weird unrelated defaults ... that your original interview submission contains no rough work would make me worry you'd commit to a suboptimal path without opportunity to reset the course after a preliminary critique.
I don't like all aspects of its design but after getting through its initial quirks the Android Simple-Workout-Log app is dead simple and bare-bones and works quite well without the busy graphics.
I am a software engineer and I was once asked during an interview at a large hedge fund to pick a side and debate why war is justified.
When I pressed them about the relevance, they indicated that they often have heated debates on all manner of topics, so they wanted to see my thought process.
I enjoy solving complex problems, but socio-ethical problems are way outside of my wheelhouse.
I politely indicated that I didn't think the company was a good fit for me.
When interviewing execs who would be building new departments a few years ago, I would pose the question of "In the wake of Arab spring in Libya, the people decide that you are their next leader. What happens in your first 100 days?".
The answers ran the gamut from lazy to fantastic to terrifying. A lot of answers where generic "coalition building" variety. The better answers identified key areas to focus on like infrastructure, basic services, etc. The best answers had clear goals and possible government structures supporting accountability.
Bad answers had the exec consolidating power and crushing opposition. The worst answers had the exec killing people to achieve their goals. Not joking, I had several answers that where "I would find my rivals and kill them".
Overall I thought it was a good question as those who performed well on it and where hired built great sustainable orgs and those who did poorly where usually shown the door within a year. Those who did well where able to take a crazy situation, break it down into smaller problems, and then solve for them while those who did poorly where usually relying either escaping the problem via committees or flat out crushing opposition.
Asking a politically charged question on statecraft and judging the answer based on your own personal ideas about how to run a state strikes me as a bizarre way to conduct an interview, even for an exec.
This is why we need less humans and more AI/ML in the candidate selection process.
The guy posing Arab Spring interview questions sounds just like those 65 year old talent scouts in MoneyBall skipping over a talented 3rd baseman because the kid “doesn’t have good legs.” Humans are immensely stupid and irrational when it comes to judging other humans.
If it's a random number generator, it's not ML. If it's unbiased, it's also not ML. ML is always biased based on the data you train it with, just as any student is biased based on the information they are educated with.
Its actually not such a stupid answer.
Its completely inhumane etc but it worked well for Saddam Hussein (among others) who conducted a purge on taking office in 1979 and then never had his leadership questioned again.
Execpt the question isn't about an organization or company its about running a country and there are a lot of countries where democracy just does not seem to be suited/desired, unfortunately.
That’s what makes the question hard. It’s what would you do in this politically dire situation to lead a country while at the same time impressing an interviewer so you can get a job.
well he was the leader for 24 years and outlasted pretty much all democratic leaders from in his time. I'm not saying I recommend doing what he did and you are right history will definitely not remember him fondly along with many other cruel leaders. But his tactic was effective and I'd say you could argue that he would have been the leader much longer if he hadn't pissed the west off in 91 and if 9/11 hadn't happened.
It would t surprise me if a lot of these stupid questions were actually useful because they indirectly measure something else. On that note, assigning lengthy interview projects is probably a good proxy for “won’t negotuate to hard on his salary”.
But as an employee, I’d sure prefer being treated like a human being...
The OP mentioned that this interview questioned worked for them. So, at face value, seems to work?
The thing is all these weird interview strategies work at some level. Its very self-reinforcing. If a company kept hiring duds, they would change their interviewing style.. or simply go out of business after hiring enough dummies.
> Asking a politically charged question on statecraft and judging the answer based on your own personal ideas about how to run a state strikes me as a bizarre way to conduct an interview, even for an exec.
Seems like exactly what you want so you don't get people who will be playing the opposite sort of political games than you want (whichever sort you may prefer).
He's hiring an exec who will presumably have to operate within his company. If the exec's thought process differs too much from his personal ideas, both the exec and the company are gonna have a bad time.
A problem is that when you topple a dictator you need to first secure the levers of power lest you be toppled yourself.
Suppose your new country has a $1bil in its budget, then spending that billion on schools and hospitals and infrastructure for the people means that there is suddenly $1bil that is not being siphoned of by the corrupt old guard that might still hold levers of power.
If you give them nothing, then they will rebel against your rule. If you try to compromise and give them $500m, they will only be loyal until someone comes along and offer them $750m and then loyal only until someone else can give them $900m. So the game is stacked in such a way that the only way to stay in power is to siphon of the $1bil on corrupt cronies and spend nothing on the people.
That is why it makes sense to purge those old enemies, but that may be difficult in itself.
I have to admit I'd give a pretty Machievellian answer as well, on the assumption the question includes having to keep the position. In harsher political climes, you can't afford to put "building a strong society" on the top of your priority list. One of the ways strong societies become strong societies and stay strong societies is to make it possible to play a lot of politics as much less of a bloodsport, so things like "building a strong society" may reach as high as priority 4 or 5 for our politicians. It isn't perfect, but it's a lot better.
Of course, the fact that I wouldn't do any of this in real life is also why I'm never going to face this hypothetical.
I'm going to guess "Abdicate power as thoroughly and comprehensively as possible to whoever seems most likely to be next in line, so that nobody is ever tempted to come after me or mine, because I am not suited for such a role and I deeply know it" would probably not be considered a viable answer by the questioner....
"In the wake of Arab spring in Libya, the people decide that [George Washington is] their next leader." is not what happened. Libya is not the 18th century British colonies.
Perhaps I'm in the minority here, but I don't like this question because you know there is a politically correct answer and then an actual course of action people would take.
The proper answer is: introduce democratic government, clean up the country, introduce proper amenities (healthcare, etc) and spread the wealth for the benefit of the people.
Fact of the matter is: absolute power corrupts, and plenty of people would loot the oil money reserves for their own benefit if they had the chance. Just have a look at what happens in Russia.
I'm sure we can find a lot of things more horrible than Iraq. Like slavery as you say, or bombing Hiroshima and Nagasaki, or the war in Vietnam...
But what I think OP meant, is that the bad things which happened in Iraq were not the intended outcome of what the US did. Those other, more horrible things were intended consequences, not mistakes.
Kind of, yes. If you're wrong intentionally, we have all sorts of other words for that, including trolling, propaganda, misdirection, incitement, lying, fraud, perjury, all of which have their own connotations for your intent. We wouldn't really call any of those situations a "mistake".
Not only did we try, but we did it the wrong way imo. When you try to pull something this complex off and completely disregard the local culture, politics, and sectarian makeup, you introduce chaos. Did anyone learn from Sykes-Picot?
Sorry but this is a very poor answer. Lack of understanding of the local culture is very evident. Every country in the world cannot have a system like the United states. Cleaning up the system without understanding it would be terrible. Just like Iraq!
Has it ever worked in that order? It seems like most current democratic countries first cleaned up the country and then introduced a democratic government. I’m thinking of recent democratic countries in Asia, like South Korea, Taiwan, and Singapore.
Or if they did start with a democratic government, to limit the voting to certain groups, like the US, Canada, and Australia did before universal suffrage.
Are there any successful countries that haven’t “cleaned up” first but introduced universal democracy?
Rulers derive power from other people. Your first job is to keep the people keeping you in power happy. Fail that, and you'll be replaced by someone else.
Democracy means shifting it from a small number of people keeping you in power to all of the people. It's not easy.
The reason Russia loved Putin is because he cleaned up low level violence and corruption. It's hard to have democracy when people are getting shot in the streets. This is something you completely fail to understand. Furthermore, it's hard to have a "democratic government" when everyone is poor. This is so glaringly obvious that it's painful reading your post.
What would your thoughts be if the interviewee, after that question being posed, asked for a 10 minute break (or thought session) to gather his thoughts about how he would act within 100 days?
- Would you discount the person for not being able to answer your question right there?
- Would you think that this is a person who doesn't like to make rash decision (or the opposite)?
- Would you be thinking that this person is wasting your time?
- Would you allow it or have 'em answer right then and there?
It's a neat idea for the interview, but it could have caused problems, at least in the U.S., regarding labor law:
Suppose the interviewee had answered, "I would work towards a proper implementation of Sharia law." or "I would start by building Christian churches and state-sponsored public preaching." or "I would start by replacing all mosques with science-and-technology museums."
At that point any candidate who wasn't hired could have a good starting point for a lawsuit based on hiring discrimination by protected class.
Ehh, only if there was a demonstrable pattern. You can't just sue an employer for not getting a job because you mentioned your religion as the basis for something.
I mean in the US you can sue for whatever you want, but the likelihood of it happening or being an issue is quite small in this scenario.
This is true. But in this specific case, it is unlikely a single person, especially one in the position of interviewing for a lower level job, would have enough money to rival the company in a lawsuit or cause them severe financial pain. Usually it's the opposite situation that is financially ruinous.
The point isn't that the lawsuit is going to ruin the company. It's that the lawsuit is going to cost the company money in almost every case where it happens, no matter how frivolous the lawsuit. The smart choice for the company is to limit risk factors for having to face a lawsuit.
Do you mean it’s a bad answer because that behavior doesn’t fit the type of person you’re looking for?
I don’t know anything about Libya’s political and social situation. For example, how many people decided I was the leader? Is the military on my side? Is there an opposition with a tendency for violence? Have they attacked before?
There are too many unknowns for me to decide what is and isn’t a good answer. But definitely, the exec’s response would be good to identify culture fit.
>Bad answers had the exec consolidating power and crushing opposition. The worst answers had the exec killing people to achieve their goals. Not joking, I had several answers that where "I would find my rivals and kill them".
You cannot placate a place like Libya without violence. Thinking otherwise is first-world delusion. It's worse - it's the same line of thinking that plunged the country into chaos under the guise of liberating it from a dictator. That's not a defense of Gadafi - it's a reality of things going from bad to worse.
Unlike others relying, I think your interview question is good. Let's break down why.
In your phrasing you asked "what happens". Did any candidates you thought were good give the actual answer: a disaster for Libya, and the person is removed from power?
That is the actual answer.
Of course not. All of your "passing" answers to "what happens" includes building up infrastructure, basic services, etc.
In other words, what you really just asked didn't match your words. You really asked: "Convince me that you are qualified to lead Libya".
Unsurprisingly, the people who realized that this is what the question was, and gave you an objectively false answer, get a passing grade. Those who did not this, or who answered correctly as asked, fail.
Those who were understood what you really wanted, and who also successfully convinced you that they're qualified to lead Libya, "built great sustainable orgs" whereas "whose who did poorly where usually shown the door within a year".
You could have done just as well asking "The company has decided we are building a social network to compete with Facebook. We put you in charge. What happens?"
What actually happens of course is nothing good. It doesn't mean it's not a bad interview question, but a good answer is not the factually correct answer.
Without knowing what kind of company you're working at, here's the way I'd answer the "the company has decided to pivot to building Facebook. You're in charge. What happens?" first based on what I'd actually tell you, then truthfully.
Answer you want to hear:
"Boy, agrippanux, this is a tough one. I hope I'd get some equity because as you know, social networks have a network effect where success means they blow big, and that's what we're going to do. agrippanux, I've long been interested in social networks so even though your company is not really in this space, I think I could really identify some of the areas people feel strongly about, including advertising, privacy, filter bubbles, and also social media addiction. We will end up having to tackle the network effect, so we will start by identifying niches where our company already has a really strong presence. agrippanux, we don't really do much direct technical support to customer's issues, so I would start by having our web page redirect seamlessly to our in-house Facebook competition that I'm building with the team you give me, as a kind of closed beta, so that we can get a steady stream of users asking us questions and getting used to the network. We can then stealthily work on the underlying technology, while testing it with a continuous stream of real user input. As a next step, we would license this answer solution to other companies. Companies will appreciate being able to have the narrative be on their pages, and by building up factual answers and technical support, we would end up building up an expectation of value and reliability - the opposite of "fake news" plaguing social media. Many early adopters are avid supports as well, and the next step will be allowing some members to answer other people's questions as well. The third stage will be introducing some company events such as pizza parties and this type of thing, in the biggest cities. This kind of approach will let companies capture their own audiences, promote those who support their products as online grassroots supporters, and be a clear alternative to the marketing mess on Facebook. Over time we are a viable social network built on the technical foundation that you have, agrippanux, in this very building at this very moment, and based on the good will and brand that your company has spent years building - and leverage that to be a successful full social network. That's why you've put me in charge of replicating a social network using our technologies and employees, and I am confident we can do it." (Then, after a beat, since your jaw just hit the floor at how awesome my answer was.) "Hey so do I get some equity if I help you take down Facebook or what?"
That's a great answer. It's not truthful though. Bam. I just got hired for whatever role you're actually interviewing me for.
Truthful answer:
"Whoever made this change in direction is an idiot. While it is nice for me to get the resources to roll out a social network 'to compete with Facebook', it's probably doomed to failure, and my top priority would be getting out of it alive. I'd probably spend a lot of marketing money getting a few million transient users who aren't actually engaged, and then jump ship to an actual social media company that knows what they're doing and has a mature product with a real value proposition. This company has no chance of creating a viable social network."
See? One of them is the truthful answer that ACTUALLY describes what really happens. Just as you asked. It's similar to exec consolidating power and crushing opposition. And one of them is the fake answer that answers the real question: "Convince me you're qualified to replicate Facebook under our company, using a few hundred thousand dollars." It's similar to the question "Convince me you're qualified to lead Libya."
> What actually happens of course is nothing good. It doesn't mean it's not a bad interview question, but a good answer is not the factually correct answer.
OP is hiring executives, not yes-men. You want people who give you the factually correct answer, because you want people who will behave ethically. Execs have to worry about regulation and corporate risk and you do not want liars in those positions.
"Execs have to worry about regulation, corporate risk, and managing perceptions both internally and externally. You do not want someone spouting the literal truth without a filter in those positions."
That's why it's a great question. It requires a filter.
I get what you are getting at, but this is a terrible question to ask.
Libya is also a very complicated place, and so it's difficult to answer without quite a lot of knowledge.
The question reeks of that ridiculous 'American can-do neo-colonialism can solve anything' kind of problem, and in that context is basically offensive.
Any answer, given by any candidate could only totally gloss over the most important in a ridiculous and igornat way.
"Bad answers had the exec consolidating power and crushing opposition." ?
This is exactly how pretty much all leaders in the the Middle East establish power. Do you think they are stupid? Or maybe that there is some kind of underlying systematic issue here?
Secondarily, is the ugly reality of the answers: you can't 'build infrastructure' while opponents are thwarting you at every turn. 'Killing your opponents' in such a situation may actually be a rational thing to do. Of course it sounds very outrageous to be talking about this in an interview, moreover, it's doubly shocking coming out of people without that kind of relevant (i.e. military) experience.
This question is just loaded with problems, I suggest you adapt it to some other more neutral context.
> I am a software engineer and I was once asked during an interview at a large hedge fund to pick a side and debate why war is justified.
Heh, I actually wouldn't have minded that too much - but in addition to being a software engineer I'm also a former Army officer.
In any case, I think such a task can be relevant, if you're working in a fast-paced and competitive environment (esp. one with a lot of non-technical staff) you need to be able to hold your own in an argument. You wouldn't want to be the guy who is always right but gets overruled 99% of the time because you're unable to persuade others.
> ...but socio-ethical problems are way outside of my wheelhouse.
Mine too, and probably 99%+ of the world's population. But that doesn't stop most people from having strong opinions on subjects they don't understand.
> you need to be able to hold your own in an argument
Then you'll hire a bunch people who like arguing about things they aren't qualified to argue about. I'd much rather have a coworker who admits what they don't know and is willing to learn about it rather than someone who is willing to vigorously argue in support of an arbitrary position.
Great point, but I could imagine this being a test for many different traits. One might be that the interviewee would be entertaining to argue with. It's anyone's guess what really motivated the interview topic.
Sounds like they've bought into the Heinlein/Rand general specialist myth (quote below) and were looking for that. Problem is, it's very easy to mistake lunatic confidence for competence, especially in an interview, and especially given the personality traits that work well in hedge funds/general finance.
>“A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.”
6 months seems too short, but in 5 years the only ones I haven't done are plan an invasion (how many people out of billions have?), butcher a hog specifically (fish though), design a whole building (designed and built a.. room.. for purposes.. no small or simple task), written a sonnet specifically (other creative writing), set a bone, pitch manure (bagged it, also bailed hay, grew food, etc.), fight physically, or die - gallantly or otherwise.
So that's 4 I definitely haven't done, 4 I kinda did, and 13 I definitely did (I count 21 total not 22).
Misaligned incentives: the people doing interviews have such indirect financial incentive to hire the right people (only that a good/bad hire affects the company's profit, which affects their own salary/stock), almost no way of telling if they made a bad hire decision, and definitely no way of telling if they made a bad no-hire decision. Remember Alan Greenspan's story of finding a competitive advantage because everyone else was happy to ignore half the population in hiring. If you have no direct incentive aligned with the business's needs, then the incentive of "I will derive pleasure from the interview process" (whether it's having a fun conversation, or having a conversation that makes you feel powerful / more confident about yourself, or hiring in a way that reinforces your existing biases) is all that remains.
> Though David Rowe, a veteran architect of the Wharton forecasting model, has played a key role in developing the firm's econometric model, the senior staff, including Miss Eickhoff, Judith Mackey and Lucille Wu, is mostly female. Mr. Greenspan explains the gender bias with the free-market pragmatism that has become his hallmark: "I always valued men and women equally, and I found that because others did not, good women economists were cheaper than men. Hiring women does two things: It gives us better quality work for less money, and it raises the market value of women."
That is, just a few decades ago, the leading economic consulting firms in the world—economists!—were regularly overpaying men because they didn't want to hire equally capable women, such that Greenspan could find an obvious mispricing in the labor market.
Imagine how much more poorly people who are not economists and not owners of their firm must be making hiring decisions on much less blatant biases than gender.
Well he is an economist so he is saying he is paying them the market rate which he believes to be underpriced relative to their intrinsic value. As an economist he would place the underpricing blame on society and the labor market. A similar situation would be if a company realizes it can get more value out of hiring junior developers cheaply over senior developers. An economist would also say they weren't underpaying the junior developers because no one else was willing to pay them more.
I think he's claiming his competitors are overpaying men by artificially pretending supply is half the size it actually is. Or looked at another way, the market value of men is determined by companies that hire everyone + companies that only hire men, but the market value of women is determined only by companies that hire everyone, so demand is lower. Either way, it's not "underpaying" relative to either intrinsic value or value in a fair (everyone is willing to hire everyone) market, which is most of the moral argument for why underpaying is bad. It's "underpaying" in the strict technical sense of paying less than what others are willing to pay men, but that only applies in the hypothetical where one single woman were being hired as if she were a man; if all women on the market were hired like men, supply would increase which would affect everyone's pay.
I think there are multiple solutions to a gender wage gap resulting from markets with irrational participants, and reducing the market value of the overpaid gender is a perfectly valid way to accomplish it. I think that's what he's doing.
By hiring women he's reducing the supply of women in the market which presumably raises their value, but I do agree the more progressive approach would be to hire women for the value he would be willing to pay men
Note that he's not necessarily willing to pay men more - the reporter says the senior staff was "mostly female," not balanced. He might have just been willing to pay men and women the same salary, which was less than what other companies were willing to pay men.
Not necessarily. He could be underpaying everyone. In which case, the degree to which women accept offers vs men would contribute to a systemic pay gap, without directly intending to.
Non-baited statement (and I'm aware it's discussed a lot here), but my experience with people saying the candidate wouldn't be a good 'fit' was just a way to enforce bias either unconsciously or consciously.
And led to problems with group-think and an inability to think in different ways.
Now granted my hiring sample is less than 100, but that has been my experience so far.
As a serious question, why do you think that dissimilar groups product faster or higher quality work? How far can you take that concept? Should they even speak the same language?
It probably makes it easier to act like a professional. If you reduce the social gel at work, perhaps more work gets done. Maybe I've gained a morning of productivity today because a conversation didn't devolve into a Simpsons quote-athon.
why should the candidate just apply for software engineering jobs? Isn't focusing on one field just enforcing their bias to a particular industry? Shouldn't we enforce basic rules that candidates apply to a diverse gamut of other jobs, such as truck driving, painting, and customer service? And shouldn't we shame them if they decline an offer for a labor job by calling out their biases? Shouldn't we require candidates to apply to more than one company? I mean, if they focused on one company at a time, that clearly enforces their bias (either unconsciously or consiously). How are we going to contain this blatant bias in job applicants?
I can't tell if you're being serious or not. If you are, then you have missed the point entirely and I would argue don't even understand what I was talking about in the first place. If you're not, then you got one over on me 100%.
It would actually be a pretty good way to see if a candidate can be persuasive, logical, able to acknowledge opposing points of view, and capable of divorcing argument from emotion on heated topics. Pretty crucial skills in the workplace!
But the problem is, that's not how it's presented. The OP stated that it was because they enjoyed having spirited debates and wanted to make sure the interviewee can hold their own (my words, not OP).
If the purpose is to do what you say, then you have to say that. If in an interview you find yourself trying to trick the people interviewing, for any reason whatsoever, you're not clever, you're a dick and your company, by extension, is a bag of dicks.
> How is that even remotely related to the job at hand?
It is not. But they cannot ask you "did you vote for party X? Because everyone here votes for party X and it would be awkward to have someone with Y sensibilities", that would be discriminating.
> When I pressed them about the relevance, they indicated that they often have heated debates on all manner of topics, so they wanted to see my thought process.
One of my favorite interviews was when I was posed the question: "you work for the railroad, we've just spent several months asking our customers how they feel about the service; the majority of them are unhappy, what do you do?".
I spent an enjoyable hour in front of a whiteboard working on ideas with the interviewer.
Yeah. I had an interview with a 2 tests. One had a you check a list of personal attributes you see about yourself (diligent, careful, fun, approachable.....) Second side of paper, same list of terms “check all that others see you as”. Then a one hour paper coding type exam.
I’ve also done a one hour codility online coding test more recently. I thought I’d hate it but frankly it’s wasnt over the top hard and kinda fun.
The correct answer in this case is of course YES, it's justified. It's a hedge fund after all. Motivate with evolutionary theory starting with Darwin and ending with Nietzsche, maybe even bring in Sun Tzu depending on interviewer.
I love that question, and may use it, or a variant, in one of my future interviews.
My reasoning: I would like my team members to be able to contribute in all aspects of product development. This includes not only engineering decisions but also work with the product managers in identifying both potential new features as well as short comings or issues with some requirements. It also tells me how narrow or broad a particular person's knowledge base is.
Let me try to put it in other words: it may tell me if a particular frontend engineer would be open to exploring backend development? What if the language stack changes, will they be open to exploring new languages, stacks, frameworks, OS, platforms. Or, will they be limited to what they know. Will they be able to just code what they're told, or will they be able to form and express their opinion about yet unknown things.
In all fairness, there have been so many articles about the craziness of that hedge fund, you should have known what you were getting into. Better to find that out in the interview.
None of the places I've worked at had me do a coding challenge. Either they already knew who I was through networking or they asked me to show them a personal project and explain how it worked. Each job was a great opportunity.
When I start looking for a new opportunity in a year or two, I may consider coding challenges to be a soft red flag in that I don't think they're worth wasting time over. I spent so many hours in the last 6 months doing various code challenges, some of which were borderline unreasonable, none of which landed me a position. I ended up getting hired by one of the top 3 places I wanted to work, and it was mainly because I had already networked with my interviewers. Maybe they looked at my GitHub, but I never asked. Regardless, coding challenges of any kind have consistently been a waste of time for me.
Unless I really really want to work for a particular company, from now on I will immediately end my interview process if they ask me to do the following:
- On-site coding challenges
- Take home coding challenges
- Whiteboarding
- Brain teasers
I don't have time to waste on that shit anymore. If having years of experience and a GitHub don't demonstrate to you that I can write code, then I really can't help you with whatever it is you're looking to achieve. Developers have to shotgun apply to many places in order to play the numbers game of getting hired, and life is too short to work an extra hour or more every night to code something no one will ever use that won't get me hired anyway. People are generally biased towards feelings rather than facts, and any of the above bullet points mostly serve as a Rorschach test for interviewers who have already decided whether they like me or not. Someone just breaking into the industry might want to go through the hazing process of hammering out coding challenges, but I refuse to have my personal time wasted.
To anyone reading this, networking is a biggie. My dad used to tell me that and I never took him seriously because I hated the idea that schmoozing outperforms merit, but it's absolutely true. In reality, networking should be better because, at the end of the day, most people can tell whether you're competent and if they want to work with you based on how you naturally interact with them. Formal interviews barely work because everyone rehearses for them and they only prove that you were able to tackle one problem(which recruiters often alert candidates to anyway).
> I hated the idea that schmoozing outperforms merit, but it's absolutely true
Try thinking about it quantitatively. Someone who spends the time to schmooze (read: invest time in developing a human relationship) has essentially posted that time as collateral. They added chips to the figurative pot, and if they were a fraud or recklessly destroyed that relationship, they forfeit that collateral (lose the relationship, lose the honor and respect they earned). So that's why it's convincing to invest in people who are willing to post collateral. Now certainly, there are people who have no shame and will pull cons. So this method isn't infallible. But they can only get so far before they get exposed, especially in our industry.
> time to schmooze (read: invest time in developing a human relationship)
There's a reason why engineers are looked upon as aliens. "developing a human relationship" is the basic thing all people do, not some annoying side requirement
Refusing take-homes is a reasonable stance but refusing to whiteboard strikes me as overly critical. It's a pretty standard thing to do in an interview... while whiteboarding may be a little silly, walking out of an interview after someone's already blocked time for you because you think whiteboarding is silly is even more silly than whiteboarding is. If I were the interviewer I'd be concerned that you can't be even a little bit flexible with things that you personally don't like.
> It's a pretty standard thing to do in an interview...
I'm not interested at working for standard companies. :) But everyone is different. If whiteboarding is someone's thing, the more power to them.
> walking out of an interview after someone's already blocked time for you because you think whiteboarding is silly is even more silly than whiteboarding is
If there was a misunderstanding where they blocked out time in an interview for whiteboarding, I wouldn't be rude and refuse in person.
What I do is ask in the phone screening or the first engineering interview about whether the technical interview would involve whiteboarding or brain teasers. If the answer is yes, I politely end the interview process. That way their time is wasted minimally.
However, I have no qualms about walking out of an interview that involves brain teasers.
> If I were the interviewer I'd be concerned that you can't be even a little bit flexible with things that you personally don't like.
Interviewing is a two-way street. How about you be a little bit flexible and not make me do something that you personally like? No?
Yeah, I didn't think so. I'll just work for companies that don't do whiteboarding in interviews, which are a dime a dozen.
Yes, that was a bit crass, but it amazes me how people in the position of interviewing believe that interviewers should care about whether they're concerned over something.
As a hiring manager I would take a refusal to whiteboard or participate in an in-person coding problem as a sign that the candidate knew they were not qualified to write code and therefore declined so as not to waste anyone's time.
I'm sure that doesn't matter to you as you don't want to work at standard companies anyway, but I have seen plenty of people fail to solve fizz buzz who had the nerve to show up for a Senior Dev role and therefore I will always ask my candidates to write code in front of me. Anyone can copy and paste their way into an active looking github profile, much fewer can talk me through their thought process as they solve a real problem that I have given them to solve.
If all I had to do was a problem like fizz buzz, I'd probably go ahead and do it. In my experience, most coding challenges that are in-person or take-home are more involved than that Fizz buzz, taking an hour or more. I can't think of a single coding challenge I did in the last 6 months that didn't involve either building a full application or something with complicated or game-like logic. The closest thing to fizz buzz level of simplicity might have been the custom file/directory system I had to work on, but even that was more involved and nuanced than writing fizz buzz.
Hey, interviewing is a two-way street. I'd love it if the candidates that know they don't want the job halfway through told me so and gave me my time back.
I am under no obligation to continue wasting your time. Who cares what the interviewer thinks here? You already (at least mentally) rejected them, so why should I care whether they would've said yes or no?
I wouldn't personally walk out for this reason, but things don't change when there are no consequences.
Whether or not 'everyone' wants to work there is not hugely material.
They, and others that do the white-boarding, are 'the best'. They have loads of talent, operational efficiency, ship tons of great products with great innovations.
White-boarding done appropriately is an excellent means to develop an understanding of people's aptitudes in a short period of time.
Here is an example of a Google white boarding interview, it's excellent in many ways:
Amen brother. Networking is absolutely the way to go. As employees or grunts and not a part of the ownership/capitalist class, our network is our biggest asset beyond our skills.
Some suggestions:
1) Try to socialize with fellow engineers at work. Coffee, lunch, grab a beer. Host or attend a dinner party occasionally. Put yourself out there.
2) Don't work at any single employer for more than 3 years unless you have very compelling reasons. Stock options or you have excellent opportunities staying put. Changing employers allows you to grow your network and your skill and experience.
> There's nothing wrong with on-site coding challenges, white boarding, and a good chunk of brain teasers.
If a company is looking for the kind of candidate that can and wants to work through those problems, then by all means they should do it. I personally don't recommend those kinds of companies because I'm unconvinced that on-side coding challenges and white boarding do much more than to weed out the really unqualified people and humiliate nervous but talented engineers.
Brain teasers don't measure anything other than that the candidate memorized the answers to common brain teasers, or as I've witnesses on more than one occasion, their ability to remember the answer given to them by a third party recruiter.
Maybe brain teasers are for you, and you can go work for companies that think brain teasers measure something. I'll go work for the majority of companies that don't fall for that idea.
> Having people code a small problem might be by far the best way to understand how they can do the job.
That can be, especially for junior engineers. If a company still wants to run aptitude tests on mid-level and senior engineers with years of experience, good references, and personnel projects, then they're being lazy. The fact that I don't want to burn myself out doing unpaid work at this point in my career isn't my problem.
Companies are free to not hire me. LOL
> Your Github code does not demonstrate what people are looking for.
And yet I've been explicitly hired because I had coded something relevant on my GitHub profile and could talk about it in detail in person. It sounds like you believe that a significant number of people with personal projects are really just bullshitters, and that making them write more code is going to tell you whether you want to work with that person.
No personal offense, but what you are suggesting represents exactly the type of company I want to avoid for good reason: Companies that think they can conveniently measure a candidate's aptitude through unproven metrics like brain teasers.
"I personally don't recommend those kinds of companies "
You wouldn't recommend Google, Microsoft, Facebook and a host of the best companies in the world because they do 'white board interviews'?
You're not only working against the data here, you're working against reason: asking people to solve problems that they may encounter on the job is an excellent means of measuring a number of things, even beyond their ability to solve this problem.
There are still quite a lot of 'senior engineers' who actually might be valuable in some specific circumstances, who can't walk through problems or write decent code at all.
This example [1] from Google is really quite good. If you watch carefully you'll see how many elements and opportunities there are for a 'good developer' to shine both technically, and also in terms of communication. And also for 'ok developer' to just do an ok job.
They're not asking bizarre or impossible to solve questions that you'd have to study for either.
"That can be, especially for junior engineers."
No, it's good to weed out 'senior engineers' who think that their experience and 'good relationships' entitle them to a job.
"I don't want to burn myself out doing unpaid work" - an on-site bit of coding is not 'unpaid work' - please don't take this angle that your (or mine, or anyone else's) bit of quick hacky coding is 'work' that could be useful really in any way.
This is different from a '2 hour but really 2 day' take home test which is ridiculous.
"And yet I've been explicitly hired because I had coded something relevant on my GitHub profile"
That's good - and I suggest in many scenarios that may be just fine, but it doesn't abnegate the fact of the matter: that 'white boarding' and 'on site coding' (specifically not big ugly take home projects) are an excellent means to measure core abilities.
"but what you are suggesting represents exactly the type of company I want to avoid for good reason"
Again, the best companies do this for a reason. If you want to avoid a massive tranche of the best companies in the world in various corners and niches then that's a personal decision obviously.
Finally - the most contentious 'brain teaser' - they come in two types: the 'tricky problems with solutions' and the more 'open ended types'.
MS used to ask the former "i.e. a kid, an adult, and a row boat owner are at the side of river and must get across with a watermelon. But the kid can't row, the boat owner won't take the watermelon etc. etc." - these I don't think are very helpful in most cases.
But the consultant type question: "how many tennis balls can you fit on a 747" - these can be enlightening. I was asked once "Quebec separates from Canada, how do you divide the national debt" in a consulting interview. Very good question. Aside from the problem of some people getting nervous, and not being very good 'on the spot' (to which I would encourage giving time to think about it), you can learn quite a lot about people in these scenarios. It's an opportunity for people to think about a problem in different ways, to apply some of their niche knowledge, and importantly to see if they can come up with a very crude, ballpark answer to a fairly ambiguous situation. This might be more apt to consultancy types of situations, but I think there's value in software as well, given the right kinds of questions.
I interviewed at Uber before they launched. Their "take home" was to create... Uber.
Here's the exact challenge:
UberCab Coder Challenge
1) Write a program that determines the wait time, trip time, rating, and fare for a black car trip in San Francisco, given that the customer's pickup location and destination is randomly placed anywhere in San Francisco proper, and given that there are X number of cars all placed randomly across the city. Use GoogleMaps API for trip duration time. Run simulation 100 times for 1, 5, 10, 20, and 50 cars. Output average wait times and trip times for each # of cars the simulation was run on.
2) Take #1 and do the same pickup and trip time calculation given that there are Y customer requests per hour (extra credit if this is done with a Poisson distribution ;)). Take into account each car's availability (given that they may have been selected to carry a customer and are in transit on pickup or on actual trip, and thus unavailable). Run simulation given the number of cars in #1, but for each simulation in #1, run once for Y=1x, 2x, 5x, the number of requests per hour (X being the number of cars in the city, as defined in #1)
3) Take #2 and then make a GoogleMaps mashup that then shows the various trips that were taken for each simulation. Provide mashup interface showing all trips but also provide ability to only show trips for each driver.
4) Write a "Hello World" app on the iPhone that for a certain driver logging in shows all of his trips taken in the last simulation, the average rating that he got, and the total fares he collected during those trips.
>Write a "Hello World" app on the iPhone that for a certain driver logging in shows all of his trips taken in the last simulation, the average rating that he got, and the total fares he collected during those trips.
That's not "Hello World" app, a "Hello World" app prints "Hello World." The purpose of a "Hello World" app is to demonstrate you've setup the environment correctly, nothing more.
Having never worked in SV, are these home assignments common there? I know several engineers who would refuse to do any homework at all as a matter of principle, so a task of this magnitude without compensation sounds laughable on the face of it.
It's not the norm but it's not uncommon at startups. Also, it seems to be standard practice for data science interviews.
I personally don't put up with it, but then there are always lots of commenters in HN who bemoan whiteboard interviews and prefer take-homes. To each their own.
I prefer take homes if they are reasonably scoped and turn around expectations are around 1 week.
I would rather whiteboard about the design of the app/tool than some random question another engineer pulled out of their ass twenty minutes before the interview and has unreasonably strong opinions about the design and implementation based on whatever source they stole the problem from because they don't actually understand the problem either.
The scope is important. Some of the data science take homes I've seen could be looking for a decent/good-enough solution that takes an an hour to a full-fledged research program that would keep a few FTEs busy for a year.
Home assignments are probably not yet the norm, but are becoming more common. There's been a lot of backlash toward whiteboard coding, but most companies still want some sort of concrete work product. I would expect a reasonable home assignment would take no more than a couple hours to complete.
If a company gave me something as time-consuming as Uber's, I'd say no thanks. Otherwise, I'd prefer a home assignment over whiteboard coding since it will give the interviewers a much better idea of my skills.
As always, there are advantages and disadvantages to both. I've only experienced whiteboard coding (and in a limited fashion), but I'd prefer interacting with an interviewer and getting instant feedback compared to submitting a lump of code and potentially never hearing back. Communication is something key they fail to measure with this type of interview.
I've never done a take-home either. I'm perfectly able to "pass" a whiteboarding interview, but I just find them kinda lame when it comes to actually evaluating skill.
I think pairing the take-home with the in-person interview is a nice compromise. Assuming you didn't turn in complete crap code, they can call you in for the interview and you can talk through your solution and why you made the choices you did.
Another possibility is to make the take-home a part of the in-person interview, and just give the candidate a laptop and internet access and a few hours to work on it, with an interviewer at hand to answer questions. The downside of course is that everyone gets less time just talking to each other. A plus is that this timeboxes the possible assignments so things don't get too crazy, and also weeds out people who can't complete it in that time (which could be bad, too, I guess).
Our current assessment is about 4 hours and i think that's too long. When i was hired I remember multiple points where i seriously considered giving up. O can't help but think how many good people we lose to an excessive applicant process
Serious question, are you not also concerned about the poor quality of worker you might take on with a less rigorous process? I think I'd accept a few false negatives to avoid the hassle of false positives (in the UK at least, where employment law protects employees above all else and people can be very hard to get rid of if they want to be).
UK has trial periods too though, or? I've been in trial periods from 3 to 6 months, which is ample time to find out if the hire was successful. It's of course still very bad to fire someone during the trial period though (for both wasted effort and employee morale), but there is that option.
FWIW, this is not how Uber does interviews anymore. Now the process involves a technical phone screen and an onsite, with coding exercises that are meant to be completed in 30-45 minutes. I believe it's pretty on par with other tech companies in the Bay area.
It's on par but still bullshit. I interviewed with Uber, got said phone screen, managed to solve the problem then was told Uber decided to "go in another direction" and would not be moving forward with my application. When I asked for clarification as to what that meant considering I solved the problem the recruiter called me and said some hand wavy stuff about how I didn't solve it "quickly" enough (I had plenty of time left to answer questions about it and ask question of the interviewer eventhough my first solution was not optimal and we did optimize it) but that the interviewer was up for having me try again. I declined.
> I interviewed with Uber, got said phone screen, managed to solve the problem then was told Uber decided to "go in another direction" and would not be moving forward with my application. When I asked for clarification as to what that meant considering I solved the problem
I understand the frustration with interviews but your comment seems to show a lack of understanding in how interviews work.
Interviews are a competition, not a pass/fail exam. Or if we're going with the exam analogy you're being graded on a curve, so you could get a 90% but if everyone else scores 95+% you didn't do well.
What matters isn't whether you solve the question but how well you do compared to everyone else.
That's a great analogy. Just a small nitpick: While it's similar, in reality, interviews are not quite like being graded on a curve in school. The main point of a technical screener is to decrease the likelihood that a candidate will bomb the more demanding onsite interview loops. This is especially important because these loops typically take up most of a candidate's day, disrupt the day of a bunch of engineers, and might even cost the company hundreds or even thousands of dollars in plane tickets/accommodation costs if the candidate is flying in from out of state.
It's true that an interviewer might look at previous candidates to calibrate expectations, but they don't necessarily pit candidates for the same team against each other as would be the case in school curved grading. Usually what happens is a candidate just barely solves the technical exercise, but also raises a bunch of yellow/red flags. A common rule of thumb among tech interviewers is "if in doubt, reject". This is - in my experience - so common that a company will typically nab the first candidate that clears the expectation bar. It's actually rare that two or more candidates would be up for consideration at the same time because it's hard to even get a single one of high enough caliber in the first place, since good engineers are in extremely high demand and are almost never actively looking for jobs (recruiters reach out to them instead).
This implies some sort of standardized assessment, which is seldom true. All sorts of bias is at play under the guise of culture, personality and team fit.
> This implies some sort of standardized assessment, which is seldom true. All sorts of bias is at play under the guise of culture, personality and team fit.
I'm not sure I understand how it does?
I just said that as an interviewer you have no idea how well you did because you can't compare yourself to other interviewers. Or, in other words, even if you somehow knew your "score" (for some made up score) that wouldn't give any indication on its own whether you passed or failed because you'd have to know what scores other people have gotten.
I agree with what you're saying about biases, but that merely affects what or how scores are given, and not whether you can make judgments about your outcomes based on score.
Sure, but without knowing what the grading criteria are (coding speed?, optimality of solution?, etc.) I can only go by what appears to be the criteria according to the signals I get during the interview ("Is there a more optimal solution?", "This looks good to me.", ...). And yes it is pass/fail often times.
I've actually had something like that happen to me too, but in a different company. From my experience as someone who's conducted a lot of interviews myself, I think this can happen due to various reasons:
- broken telephone (recruiting org rarely physically talks to eng org). If you call the recruiter after the fact, they may not have the context for the rejection and might make stuff up on the spot to get you off their back, because at that point, you're kind of "wasting their time" (given that their job description is to keep the hiring funnel greased, not maintain long-term relationships)
- technical interviewers don't always have interviewing training and may reject based on bogus reasons (e.g. feelings), then try to "justify the decision" after the fact. A few might not even keep good notes of their interviews
- sometimes candidates do solve a problem, but are legitimately rejected because they did so in a non-ideal way (e.g. performance problem, missing important edge case, struggling too much with basics like syntax, soft skill red flags such as lack of interest/proactiveness, etc). But often times the interviewer doesn't give the negative feedback to the candidate, leaving the candidate with a false sense of accomplishment.
I'm on the other coast, not even in a major tech hub, and we do a take home assignment now to screen applicants.
They choose from 2 languages,and an experienced coder is done in 5-15 minutes. A fresh grad maybe 45 minutes.
The tasks have no practical work application, and are just code samples.
If they pass this and a phone screening, there is a live challenge as well -- the take home version we give people 48 hours (their choice of weekday or weekend). In person is time boxed and part of the interview, but is the very last stage and usually simple functions to rule out people that needed help to get to that point or simply had someone else do all the work (which we've already had come through our small 10 person te team)
How many steps total? How long to get through the process? This may work for lower demand markets but sounds excessive in competitive regions.
We've been super successful in mining the late round rejections from companies like Google. Getting to an on-sight is a huge signaling factor, they do the early filtering and we can move fast to an offer
I am not a coder. I am a very experienced manager/director/PM OF development/DevOPs/IT teams.
However, prior I was in architecture.
And starting out as a draftsman, it was extremely common for architectural companies to tes out your drafting skills on AutoCAD with a timed test.
I recall the two I took:
In Redmond Washington in 1994, I was given a test to draw out a site plan for a chevron gas station, whom was their primary client.
I was given 15 minutes. I drew it in 12.
They said that not only was I the most accurate, but the only person ever in the history of their company to finish it. [became a core designer on what was then the 'revolutionary' designs of marriage of gas-stations and fast-food restaurants... [[Soul killing]]
I got hired... but then I became CAD manager - and then just decided I liked "computers" and not arch... so I then moved to SF. So I moved on to hospitals... gosh...
... (became IT manager of a small firm on the peninsula)
So then I worked for a design firm, and they also tested me upon interview... in CAD...
Isometric datacenter cabling plans....
I wasnt really familiar with plane changing in AutoCAD at that point so I did it all manually designed based on switching my orthos for each thing. Finished a complete cable tray design in the allotted 30 min window.....
Got the job - which resulted in me being on the core data-center design team for the Lucas Presidio project.
(and I have many other storis such as this)
---
Challenges are good which are practical to the company's/teams goals and needs -- but bullshit if they want you to design their whole shit for them.
Now get off my lawn....
----
Oh I forgot to mention - now I am director of tech for a cannabis org... GET THE FUCK IN CANNABIS TECH.
I’ve personally seen several of these. I’m tempted to reply back with my hourly consulting rate because it’s crazy that these companies would expect someone to do a project like that uncompensated.
We've used them, but these huge examples give them a bad name. Some people (myself included) are terrible at whiteboarding and we'd have lost some great engineers in screening if we had only done CS quiz/whiteboard.
They are great for probing real-world development, (should) take less than an hour or two, and based on what the interviewee says they are skilled in.
I've had a few. They are usually things you can do in a couple of hours. I personally prefer it to a phone code interview. "Write Uber", though, is pretty egregious.
The worst is when you do the task and then they just don’t respond.
Not even a simple, “we’re not interested”. No feedback.
Just. Left. Wondering.
It’s cruel. It’s demeaning.
Tanium was especially cruel. 6 hours of live coding interviews and the only reason I got any response from them was because I had to consistently hound them for a couple of weeks and ask a friend that worked there to internally ping someone.
I can't tell you how many times I've never even gotten the courtesy of a response after interviews...even though I've checked in with them to find out where they were in their process.
The other thing that gets me, is not listing what the expected pay is upfront (and whether or not it's remote) on the job posting. We all work for pay and we all have a pretty good idea of what income it takes to live in your current (and future) circumstances. If your pay is nowhere near that expected range, why should I waste your time and why should you waste my time?
I got hit up on StackOverflow to apply for a company. After looking at their job listing, it said (limited remote with approval), making it sound like you had to negotiate for remote work. This is what I responded with, "I currently work remotely and have no intentions on moving. I have a proven track history of performance and productivity working remotely for over 7 years and it's not something I'm willing to give up or negotiate. So I can save us a lot of time by being upfront with you about that."
>it said (limited remote with approval), making it sound like you had to negotiate for remote work
In all fairness, that's pretty common. For a lot of positions, companies are fine with proven employees working from home part of or most of the time. But that's not the same as signing off on 100% remote from Day 1. If that was the case here, I'm not surprised that they took your response as a polite "not interested" and moved on. In my somewhat limited experience, recruiters usually aren't that inclined to chase after people who have pretty much indicated they're not interested.
Had one with Deliveroo. They responded, but they were declining me because I didn't do what they explicitly told me not to do in the instructions. They wanted people who "went above and beyond".
If above and beyond is required, how can it be above or beyond?
Above and beyond can be nice, but the whole point is that it is exceptional, in every definition of the word. If all individuals went above and beyond all the time to fulfill expectation to do the same, that sounds like a recipe for a PR or legal disaster because rational limiting judgement is not applied, or intentionally suppressed, in a scenario where somebody is in over their head.
Everything about the interview process is a crapshoot. My last run on the interview circuit had a few like this too, where the "code challenge" is either intrinsically contradictory, or where the interviewers give candidates a "bad score" based on their own failure to understand/process the challenge they issued. Lots of amateurs out there with a "naive cleverness" about this stuff.
Going "above and beyond" in a coding assignment can be a slight negative when I'm grading. Albeit this is a security company and adding pieces on top of the requirements could lead to unthought about interactions and more surface area.
But I'm also about work life balance, and honest day's wage for an honest day's work, so I probably defer from that company in a few ways, lol.
A company in PA called Monetate did this to me, on a smaller scale. I did their "leetcode" style test which wasn't difficult at all but still time consuming. After 2 weeks of no response I reached out and they told me they're interested and then I heard nothing for another month. After several followups I gave up. 6 months later I got an email asking if I can send over my resume (that I sent before doing the test)... it's infuriating.
I had an interview with a company that's mentioned from time to time here. When it was the founder's turn to grill me, he spent 90% of the time staring at his iphone while barely glancing at me.
Not sure about this case, but you usually get compensated for these take home projects. They also give you an amount of hours you should work on it and pay you (One of them I did was ~$200/h)
My interviewer was sitting in a room full of people when they called me, asked me questions and then proceeded to have conversations with the other people in the room without listening to me and would ask me to repeat my answers. And do it again.
15 minutes of this and I hung up on them. This was about 4 years ago and the equity that I have in the company I joined instead is on an upward trajectory instead of what Uber just did. Phew.
Ouch. I'm glad you cut it short. I had a variation on that.
Earlier in dotcoms, I wanted to work in a then-prominent organization. The technical phone screen went well. When they asked what I'd like to work on, I said something like HCI-ish R&D (in which I also had experience) for their important product, so they scheduled me for another phone screen, with the person who was in charge of that.
I'm reasonably good at hearing tone, and the person on this second phone screen didn't seem to want to talk with me, from the start. They asked me to talk about ideas, then they muted their end. I knew this wasn't going well, but I really wanted to work there, and I was trying to figure out how to salvage it, and I had to start talking immediately, so I started going into some applicable research ideas/approaches. Still muted, no comments, no questions, no backchannel cues whatsoever, so I kept going. When apparently the time block for the interview was up, he un-muted his end, and said something like, "I'd hoped you'd talk about a GUI enhancement to <product>." And that was the end of the interview.
That gave a bad impression of the organization, that they'd have someone who'd behave like that, managing people in a key group. So I didn't pursue things with the original group, which liked me.
It's a little validation, but no consolation, that their market-leading product later got driven into the ground, losing to an innovator who entered the market.
But that no one shut down the asshole, and that this was allowed to continue happening in an interview situation means that in addition to an asshole guy, there are also a bunch of asshole employees who don't care enough to speak up, or would rather have side conversations with the main asshole guy.
Especially when in a new hire/interview situation, HR is absolutely involved. So the HR department is also full of assholes who won't enforce the policies of the company.
Interviewer was not a guy and would have been my manager.
I guess I'm glad it went this way for other reasons. It was for an entirely new team within Uber to build something very specific that nobody had done successfully yet (though there was a paper about it...it's security-related). The manager was an external hire and the entire team was to be as well. This was a big warning sign to me that management didn't care very much about this team or if they accomplished what they set out to.
4 years later they still haven't delivered on that feature yet.
That is excellent point, I really like where you are going with this. It shows maturity of the applicant as well.
Sometimes I would like to say, you know, I like to work with good devs, so if you don't mind, I would like to ask you some coding questions. It freaked out some people with that, but it is in the same range as the questions they are asking.
In fact in many abusive situations the abuse is how the abuser keeps the abusee from exercising their freedom to leave. The relationship being nominally voluntary has nothing to do with whether it is abusive.
I agree in this context, but not in the broader sense. Not giving clear responses to interviewees is unprofessional and shouldn't be tolerated, but "abusive" is a bit much.
Did you complete this? To me this sounds like they were not looking to hire but would hire someone exceptional who goes through this task and does better than their own team did.
I got a take home assignment from a large company a few months ago, it was the whole shebang as well. Build an iOS application with full test suite, several screens, should work with their test-api, should be polished, offline/online work etc.
I estimated it to take at least 2 weeks. Sent it to a colleague and he estimated it the same. In reality you had 48 hours to turn it back in. I gave up halfway into the project.
Sometimes that's intentional. As a hiring manager I find value in seeing what someone chooses to prioritize when given an impossible task. If you have 48 hours to complete a 2 week project, what portions do you choose to complete? Can you explain why you chose that?
I also don't think it's cool to do what I just said without letting the candidate know that's what is happening. Otherwise they might just get frustrated and decline to work on it.
I've been a software engineer at Uber since 2015 and have lead over a hundred onsite interviews and countless phone screens. In four years I have never seen a take-home exercise be used. Every other interviewer also holds a strong negative opinion of them.
It's possible we were doing this many years ago, but that certainly isn't the case now.
If that's how you want to test potential engineers, I don't necessarily see a problem with the task itself. Just pay them! I'm sure Uber can find the budget for that.
I'm sure they're fine with engineers who don't ask to be payed for random tasks encroaching their private life.
Snark aside, it can iffy to pay someone for a task at your company when they are still under contract in their previous job. In that sense, while your point is sensible, it can be difficult to go that route from the candidate side.
Yup. Will anyone who cares find out that a competitor paid you to do some task? Probably not, but it seems like an iffy proposition to violate your current employment agreement to have an interview.
I once was invited to interview for a startup to work with their (mostly remote) team for a week, for which they would compensate me at a decent rate. I never managed to find the drive to take a week off for this, but yes, it would have been tricky to explain to my then manager.
I doubt they had the budget for that or they likely wouldn't have been essentially trying to get someone to making their MVP under the guise of a new hire interview.
Keep in mind the OP mentioned that this was pre-launch, when they were still UberCab and only had aspirations to be a peer-to-peer cab app. Long before they started marketing themselves as a logistics juggernaut or world changing self driving car creators for valuation justification purposes.
One thing I've tried at my company is to present a complicated task with many parts and considerations, and then explicitly say that you are not required to complete all of it. Just do the the minimum required (which can be done in an hour or so) and the parts that catches your interest. Looking at which parts the prospective engineer completed, we could also determine which part of the stack he/she likely wants to work on.
If you've never had a job as a developer before and don't have any open source contributions, then that's actually a pretty good take home project. Yes it will take a week or two of unpaid work, but no one is going to hire you for your first job without having built at least one project of that size anyway.
For an entry-level fullstack iOS developer it's not that bad.
For context, for my first job as a developer the take home assignment I got was creating a web app that logged realtime CPU usage and created a graph of that CPU usage that was updated in real time.
And this all had to be done using a specific language, web framework, and database that I had never used before.
>For context, for my first job as a developer the take home assignment I got was creating a web app that logged realtime CPU usage and created a graph of that CPU usage that was updated in real time.
That's at worst a 4 hour job to do with a beer over the weekend. You sound incredibly incompetent if you can't tell the difference in difficulty between that and that they asked OP to do.
That assignment is 1/128th the work of point 1) from OP.
You download a list of every street address in San Francisco, pick two at random, and write one line of code to get Google Maps to tell you how long it takes to drive between them. Where exactly is the complexity?
Op missed the part where they need to batch run their solution 200 times over and pick the quickest root. Like always ignorance begets confidence. That's not even talking about part 2 which you need to completely refactor the solution from the previous problem.
You sound like the type of engineer I get hired to replace at three times the rate when the project is on fire because they didn't understand the spec.
Is there a reason there's no naming and shaming going on here?
One company asked me to do an implementation of the Bay Area MUNI network which fetched live updates of the locations of all buses every 10 seconds; there was more to it but I can't remember it now. This was to be done in React with D3 and no additional libraries were to be used to help with integrating D3 maps in React.
The role was junior level, the job listing did not specify D3 as a requirement (which pretty much guarantees a lot of people would start into it without realising maps are one of the more complicated things to do in D3 as a newbie or how poorly D3 played with React), the role was in Europe.
If someone can confirm I'm not breaking any rules I'll happily share their name.
The best interview experiences I've had are with groups who just went through my (not very impressive) Github. A small coding challenge can be good as a thing to talk through during the final interview but anything longer than that and I'll tend to pitch sending them something I wanted to make in my own time regardless.
> Is there a reason there's no naming and shaming going on here?
No, there is no rule, unless you signed a contract stating you will never disclose any interview information, and even then, they'd have to find your post, find out who you are, and then take you to court over an extremely shaky premise.
Just name the damn company people! You're giving companies far more credit than they deserve in these situations. I'd argue you owe it to other people who may follow in your steps to know when a company is exploitative, deceptive, or just plain corrosive.
No courage in that! The true courage is in admitting I was stupid enough to take a stab at their challenge.
The interview itself was a whole other tale of awfulness but I'll limit my overview of it here to saying that they didn't bring up the coding challenge once despite being repeatedly asked for feedback before, during and after the interview. Maybe they felt awkward about saying how crap they thought it was? Be pretty weird to bring me in for an interview in that case though...
Awesome to read about you turning this into your own startup.
One thing I came to think about regarding the "4-5 hours" and throwing 5 working days on it, is that, maybe as an interviewer it is quite obvious that you put more than 4-5 hours into it and what they were "really" after were the trade-offs you would have made if you put 4-5 hours into it. Just a thought.
Not Google, but similar company and can confirm. If you spend 5x the amount of time we suggest, it’s obvious and the expectations scale a lot, lot higher than if you just spent 5 hours.
Yeah, it may translate into, "well, if I work with this person, are they going to spend way too much on details that I don't need and burn themselves out just trying to please me?" Part of working is being able to be assertive and say "ok, for only 4-5 hours, here's what I was able to do for you, so now let's figure out next steps".
Yeah, I hate hiring people who are interested in what we are doing and are excited to work on it.
It's one thing to accept that some people are busy and shouldn't be punished, a bout
are we supposed to only hire people who have zero excitement for the industry and job?
That wasn't really my point. My point is that when you are working on a team, it's a collaborative experience and if your superior asks you to do a task a certain way, there may be a good reason why. The exercise here may have been to see what ideas you could generate about the user experience and less about aesthetic polish of a final product. I got the sense from the post that the author spent a lot of detail in making it look aesthetically pleasing, which is only one part of user experience design. A larger part of it is design something easy to use and understand.
Sure, feel free to spend extra time on the take-home assignment to signal to the potential employer you are passionate, but that's a different thing.
This makes no sense for "product design". Just thinking about it, looking at existing similar products, before you even grab your napkin, takes 4 hours.
I recently completed an interview junket. One of the companies I really wanted to get into gave me a take home. I completed it in the time they said it would take, submitted it, and then got bounced out. My inside source said that the submissions are run through an acceptance test suite (not exposed to the interviewee, of course) that can automatically bounce people out of the process with no or little human intervention.
That doesn't sound fair unless the acceptance test suite is available to the candidate before final submission. In which case, depending on how the infrastructure is set up, could be expensive or slow. Seems to be a lose-lose situation.
Why not do that? Coming from the other side of table, an immense amount of engineer-hours are devoted to the interviewing process, including code review on take-home assignments. Thus we try to make the process efficient. Typically there's a set of unit tests provided in the assignment to help the interviewee, and a hold-out set of unit tests that gives us a score for basic correctness. The score, albeit a low bar, is a useful threshold for eliminating candidates unlikely to succeed in the process. Those above the threshold have their code passed on to an engineer to review beyond basic correctness, and move on in the interview process.
OP's design is much nicer than this. Maybe the "3-5 hours" requirement was hinting that it needed to be as "material design generic" as possible. Which, this design clearly does.
I disagree. The OP's app is hard to read and has way too many visual elements that are distracting. Initially it has a "wow" factor but after repeated use, those aesthetics are going to be ignored and all your left with is hard to read text and distracting images.
Google's app tends towards the bland, but I don't think OP's designs are a huge improvement.
Also OP's app is targeting the gym nut. Google's is targeting the average person who is looking to just do any type of physical activity.
I once completed the task sent to me by a prospective employer, and then thanked them and said I wouldn't be submitting it, as it told me enough about the way they worked to know I wouldn't enjoy it.
Never forget that the interview process is two way ...
Those few big companies probably have more than enough candidates. If you have too many candidates you need to adjust interview difficulty to still hire only as much as you need. I don't think these companies care that the task is too difficult for 99% of people, or that there are smart people who are not motivated enough to complete it as long as they can get as many smart and motivated people as they want.
That said, finding a job in IT if you are good at what you're doing is a dreamland currently, so declining if you think you won't enjoy working for a given company seems only natural.
Given dynamics above I think you're likely better off not working for one of those top few companies if you don't care about prestige and long term job stability.
Then again, what the heck kind of use do they have for developers that can't push back on unrealistic demands? None, unless they're in the business of missing deadlines.
This particular company was neither "big" nor one of the "top" companies to work for. I strongly suspect they were getting noddy developers and were happy with them. The reason I didn't want to work for them was categorically not because the test was too hard, it was a trivial spring-boot app. I didn't want to work for them because of what it, and the test process, said about them ...
I have never heard the term “noddy” but assume it is a cognate for “nodding off”? That would make sense in context but I wonder if you mean something else by it?
To save anyone else from reading all the irrelevant parts:
> Noddy being associated with small children's reading has led to "Noddy" being sometimes used as an adjective meaning "petty or trivial" (compare with "Mickey Mouse"), for example, in computer programming: "This simultaneous linear equation subroutine crashes out on the Noddy case when n = 1, but otherwise it works." or "Remember to check all the Noddy cases."
Some companies do technical interview great, but there are a couple things that I have noticed that always make me question if I actually want to work somewhere:
- Being sent to a website to do a take home coding exercise, with a checkbox basically saying "I won't use the internet, stack overflow, or similar". I get that you need to test my knowledge, but shouldn't you care more that I know how to search for my problem and understand a fix instead of flailing around?
- (this one I have noticed in person) Being handed a thing to build, being told I have a limited amount of time. That all they want is a proof of concept, but the tooling they are asking me to use go completely against the idea that it is a "proof of concept" since they make certain assumptions about the end goal. EX: deploy something with terraform but we don't want you to build an AMI or anything like that so just manually go into each system once you create it
I hate companies that act like it is a sin to use Google or Stack Overflow when coding.
When I hear this BS, then I know it is clear that they have never coded themselves. I would love to meet a software engineer who truly can sit in a padded room and write code with no internet access to use for help or reference.
Especially those of us that juggle multiple languages (full stack). Sometimes I know I need to run a method but I can't remember the actual method name or how it is formatted in this language. I know it exists, I have used many times before, but I might not remember the exact way it is worded. So I have to look it up. It adds <60 seconds for me to find.
Or I might know a method exists but not sure which parameters it accepts. I might usually use the first 2 params and leave the rest, but this specific use case requires changing something i don't normally need. So guess what, I have to look it up!
Sometimes I might get an error message that I don't recognize, so a quick look at StackOverflow reveals that a configuration item is off, or something was formatted incorrectly. I often can understand my problem with only just reading the first few lines of the selected answer on SO. I might be able to solve a bug in 60 seconds by using Stack Overflow, that might otherwise take hours to do without them.
Especially as software developers keep getting asked to know more and more stuff, we have to rely on documentation and helpful sites in order to do our jobs. It is crazy to expect that you don't use those resources.
Oh and ObjectiveC/Swift developers have it really bad. Some of the method names are entire structured sentences.
You're right about ObjC/Swift. Even though that's my line of work, I use C++ to answer questions in interviews. There's no way I'm going to remember method names for those.
I did have a Google phone screen where they asked me to live code something in Objective-C, but they gave me an existing class whose code I had to fill in and they were lenient with remembering common method names.
Once I had a half day exercise that went even further - they provided the computer, removing the internet, but also removing all of the normal module documentation, and for good measure forbidding you from using any documentation from any source.
There were a lot of compilererrors on the way to a solution for that one.
This is a really great marketing example! The creator has combined a top page HN post with a PH launch at the same time with really polished looking apps & landing pages.
Essentially this will give them a huge traction boost on the app stores and a bunch of new users. Super impressive.
It's definitely good marketing, so kudos to the creator. I will say though that so many Medium articles I read now are essentially marketing pieces. They'll cover some topic very, very lightly and have a call to action at the end to view the author's product or company.
Link poster here: I am not the creator of the app and don't know its author Andrew Burton. So there is no intentional coordination. Heck, Andrew may not have yet noticed that his article is a top page HN.
I have just stumbled upon this article in a UX mailing list and thought it'd be interesting to share it on HN.
I recently was given a toy problem in an interview - read data in some format, process it, and write the output. Expectation was 4 hours of work, which is pretty close to what it took me to get all the edge cases, test suite, requisite documentation, and VS project/solution files.
The feedback I got afterwards was "We're passing on you as your code doesn't compile/run on Linux", a requirement that wasn't specified.
But on a serious note, I really don't understand why they couldn't have written you back and asked if you could make it compile on Linux and send it back.
>The feedback I got afterwards was "We're passing on you as your code doesn't compile/run on Linux", a requirement that wasn't specified.
I have to ask, was there something that implied that it should specifically run on Windows? If I got a software spec that didn't spell out exactly which platforms needed to be supported, one of the very first things I would do would be to ask for clarification.
It still is IMO and rejecting someone because their .NET code didn’t work on Linux without specifying is like doimg the same for a Ruby developer whose code fails on Windows.
I guess I have a fundamentally different view of software requirements. Even before dotnet core even existed I would have explicitly confirmed that the company wasn't using Mono if they didn't spell it out for me.
Mono's use was very limited because it was an out of date and slower implementation of .NET. It's used in such niche contexts (Xamarin and Unity) that it was easy to determine if the team you are applying is using MONO or Framework.
.NET core changed the ability to make the Windows assumption. Almost everyone has received the memo by now, but there was a period of time between .NET core 1.0 (June 2016) and the release of .NET core 2.0 (August 2017) where the platform was changing so much and the performance wasn't as good as framework that anyone without a simultaneous interest in Linux rightly ignored it. After the release of .NET core 2.0 the entire community began accepting it as a faster platform that runs comfortably on cheaper servers. A larger and larger portion of new projects have been running .net core. Now, with the planned release for .NET 5 next year .NET Framework is dead. However, if you were interviewing with a .NET team before June 2016 even asking the question "What operating system will this run on?" would have been a sign of incompetence. Of course you'll be building .exe and .dll files. Of course the webserver is going to be IIS. Those were the only things .NET was good at building.
> However, if you were interviewing with a .NET team before June 2016 even asking the question "What operating system will this run on?" would have been a sign of incompetence.
It really depends on how you're asking the question. "What OS will this run on?" is going to be perceived differently from "Are you expecting anything unusual like Mono support?".
The OP didn't actually say that this was a .NET application, but if it was the answer to either question would have been important. If the answer had been that the company expected a standard Windows environment, I don't think it's likely that the second question would be perceived negatively.
Can't help but agree. If someone completed a task in anything remotely sensible then I'd evaluate it to the best of my abilities in good faith.
Of course I'd be a little surprised if they chose to use a language not listed as part of the job requirements but if it looked reasonable it wouldn't stop proceedings.
> They presented three design challenge options to pick from, with a weeks notice and advised that we should spend no more that 3–5 hours on the task (wink, wink)
I know this self-selects for people willing to do extra work for no extra compensation.
However, let me play devil's advocate for a minute as a thought experiment. I would actually worry that this issue is also a self fulfilling prophesy. If I'm told to do X in Y hours, but it takes me 10 x Y hours perhaps the employer really does expect it to take only Y hours. Maybe the typical workload at that employer is for someone who really can do X in Y. As long as that is properly compensated, I see no problem with that. If the employer is asking for an extremely high caliber developer, and will also compensate as such then it's no issue. However, an average developer applies and thus has to lie about what they can achieve in that time-frame. Let's say they get the job and then when the workload is dumped on them it obviously takes longer than the 40 hour work week and they cry about "self selecting for people willing to work extra for no compensation." When in actuality, the employer expected, based on a false presentation, that it was within the average developer's abilities.
Of course, I'm being somewhat satirical and without knowing the exact position and compensation being offered (as compared to average) no one can say if this is the case.
What a great launch story! I don't mean to reduce your experience to a marketing gimmick but I know that you intentionally crafted the story to be impactful and to coincide with your launch. Most importantly, though, you pulled it off really well!
I can't help but admire the way this was done :) I hope my launch story is this good.
People can tell whether you spent 5 hours or 5 days on a task.
I wonder whether that's the reason for him not getting the job. Likely, the interviewers wanted to see what this person is capable of doing in a short timeframe, how they'll prioritize and deliver under such constraints. This guy failed to do what they asked for.
Is HN truly this bad of a bubble that everyone thinks interview tests are normal?
The last time I had to do a test it was on site and I was just out of college. Most interviews just talk about the challenges of the team, my past work on similar problems, and how I solved them. I've been on the other side of the table too. It's usually easy to tell if the applicant is a quack or if they are skilled by how they talk about different technology: commiserating about familiar pain points, how they solved common problems. I almost feel at this point that it's unprofessional to hand out tests to people.
> It's usually easy to tell if the applicant is a quack or if they are skilled by how they talk about different technology: commiserating about familiar pain points, how they solved common problems. I almost feel at this point that it's unprofessional to hand out tests to people.
The problem with what you're describing is that it's using a proxy (discussion) to vet a particular skill (ability to program).
It's very easy to vet someone's programming skill by having them program, and it doesn't take much longer than a wide ranging discussion, so why not skip the proxy and test what we're actually hiring for?
Should a programming test be several hours (or several days) long? No. But an hour or two, I can't see how that's a terrible imposition.
When I was an industry analyst, I did a lot of writing (and still do). We absolutely needed to see writing samples from job candidates. If they had articles and so forth that had been published, great. But there's no way we would have hired someone without evidence that they could write.
> Should a programming test be several hours (or several days) long? No. But an hour or two, I can't see how that's a terrible imposition.
Agreed. However there are multiple comments discussing "5 hour assignments" and "take home quizzes", which I find to be insane, frankly. But if it's worth it to you, go for it I guess?
Maybe just ask applicants to spend an hour or two onsite without internet making them to solve project euler-type problems and discuss their solutions afterwards.
I’m sure you can somehow include technologies relevant to the position and keep the problems to the same scope.
Or these companies could begin creating actual value instead of just draining it from the world. Why don’t they expand their pipeline that converts intern/junior positions into more senior positions? And make it a big enough so they don’t need to hire as many ready-made 10x engineers from outside.
> Maybe just ask applicants to spend an hour or two onsite without internet making them to solve project euler-type problems and discuss their solutions afterwards.
I 1,000% do not care about someone's ability to solve a random programming puzzle without the internet.
Again, this is a (bad) proxy for the skill I'm actually trying to hire for. If I'm hiring a backend engineer, why would I ask someone to implement QuickSort? Wouldn't it be better to give them a realish application and have them add a feature or fix a bug?
> Why don’t they expand their pipeline that converts intern/junior positions into more senior positions?
My company has a very healthy pipeline of interns and junior developers. In a year or two, they'll be very good. In five to ten years, they'll be amazing. But I also have shit that I need to get done by the end of the month, and I need people with years of experience to complete them.
And regardless of which level I'm hiring at, I want to see them code, and I want it to be in an environment as close to how they will be working as possible. Google, Stack Overflow, an IDE, a build system, unit tests ... I want to see how they will perform as Software Engineers, not Euler-grinders.
Yeah; I think the whole "coding tests" thing is very much a Silicon Valley thing (which has of course been adopted by some non-SV companies, but its not the norm). I feel bad for HN readers who live there and that's all they know.
The best advice I've ever heard about interviewing is that you need to mentally reframe it from "I need this job" to "they need me, convince me to work here". A big part of this is building a network and finding jobs through that (which is how everyone has always found careers, mind you); when you're introduced positively through a third party the team trusts, the conversation inevitably biases more toward the team hunting you, not the other way around.
And if you can't do any of that then you should probably be happy that companies do big project coding/design interviews that give you a deep chance to show off your skills. Over half of what teams are looking for in candidates isn't the hard skills; its social aptitude, teamwork, networking, the soft skills. If you don't have a network and had to land this interview via a lowest-common-denominator recruiting page or email, then at least you can dazzle them with a great project, but you're already starting out behind.
My company tends to hire more junior developers, I've been pushing for years to raise our requirements for certain teams, but its a cultural understanding problem from people at the top. Coding tests are the only strong indicator I've found for the interviewing process. I've had candidates talk their way through all of the other steps, only to find they can't code their way out of a paper bag. I really think the only accurate way to tell how well someone codes, is by looking at their code, and a coding test is a good way of not only doing that, but also comparing code from one developer to another in a controlled manner. I'll take it over hackerrank and other similar shitty websites any day.
For my internship I first had a good conversation with the lead developer. After it was determined that I was a culture fit and didn't bullshit them, I got a small simple assignment which I had to do on a whiteboard. Later found out it wasn't really meant to figure out whether or not I could write a simple function, but more to see what my thought process was. Like, did I instantly start writing one big function that did stuff - or did I break the problem down into smaller steps.
Anyways, the interview for my second job (for a fairly big consulting firm) was purely focused on culture fit, and questioning about stuff I've previously made.
Third and current job was the same as the second - only on my initiative I came by on another day to check out what the other developers were actually doing and trying to help them out.
I don't think a test is really useful, if I'm not up to par (or they don't meet my expectations) either of us can decide to end the contract in the first month. This focus on tests strikes me as silly, especially in the USA, since you can just be fired for no reason anyways.
edit: I do have an (outdated) github repo with some code, but not much else.
My team does a take-home coding test and we've found it to be very successful. The people we've hired as a result have been great and actually told us they preferred it to other interview methods they've encountered.
The challenge is coming up with a standalone task that demonstrates the skills needed for the job while _actually_ requiring the amount of time you say it should take. Seems like most commenters here are upset that a "2-hour test" really takes 6+ hours. That should never be the case and I wouldn't want to work for that place either.
Another key part of this is should a candidate pass the take-home test portion of the interview, the in-person interview should be fairly quick and mostly a judge of fit/character. It should not be loaded with more tech questions. The reason this works best for everyone is the interviewee doesn't need to waste a vacation day on-site unless they have a really good chance of getting the job.
I wish it were so, but it's not. I usually start interviews with 5-10 minutes of talking about a previous project of the candidate, digging in to see how sophisticated the work was, how they solved problems, etc. I've had plenty of examples where someone bullshitted well enough to pretty much convince me that they knew what they were talking about, only to have it become clear in the more hands-on part of the interview that they couldn't possibly. I admit I'm not a great interviewer, so perhaps a really good interviewer could tease those things out, but I maintain that it's not always easy.
I think the usual situation is that the candidate was on a team where some impressive work was being done by others, understood it enough to talk about it in detail, but didn't actually do any of the challenging work themselves and couldn't work at that level consistently.
I'd like to think most people aren't actively lying or bullshitting, but it's easy enough to implicitly take credit for work done by a team when you're trying to sound impressive.
Just my two cents, but as a candidate I much prefer the tests to a whiteboard. I've had interview tests for a majority of my recent (senior level) positions in NYC, but the questions generally aren't to this extreme, certainly not basis for a new startup level problems.
More like "we've laid out an API contract, implement these 3 endpoints in a language you like along with these few extra small requirements and some unit tests". Usually takes 2-3 hours and I think it's a more fair assessment of my work and at least I get to use my own environment and work on my own time with tools I know rather than with someone sitting over my shoulder and a marker. It also helps drive the discussion during in-person interviews away from the binary tree puzzle type questions and focuses more on design decisions and other considerations I took in to account.
Why is Googleʼs design so bad then if they have such a tough interview process? Corporate structure/culture kills even best-of-the best talentʼs initiative? I think, what they need is non-conformism. But they donʼt select presumably for non-conformism during interview. They select for obedience. (Just thinking out loud)
When Google (having so much resources) will release a product whose UI would make people say, “wow?” Not just the (superior) data-processing capabilities or pretty good hardware.
Clearly this is subjective, but I'm regularly impressed with Photos and Maps. Thoughtful, subtle, and yet informative. With some delightful interactions.
Google products are optimized for action, not impressing Dribbble passerbys. Although, a "wow" UI doesn't have to be mutually exclusive with the goal of completing actions, but the who's to say others aren't saying "wow", even if you aren't?
I really despise these take-home projects. In NYC it's not uncommon for a developer to make ~$60 an hour, and these take-home projects can take anywhere between 2-6 hours, meaning that these companies are expecting you to spend up to $240 on a task where there's not even a guarantee of getting the job.
Whenever I've done these, I usually end up doing them in JavaScript in a hyper-functional style (as is my nature since usually they won't allow me to do them in Scheme or Haskell), which ends up with the reviewer asking me to write it in a more "modular" way (which really means "please write Java in JavaScript").
I have a line in the sand now that I'm done doing take-home projects; I really don't need to donate my after-work time just to have someone who doesn't understand functional programming explain to me how it's not as modular as some Java OOP crap, especially when I could spend that time building a project that will, you know, actually make me money.
EDIT:
I realize that this came off as overly-hostile to Java. I'm not a fan of the language, but my point wasn't really the bash it, as much as complaining about people acting like my functional code isn't good because all they know how to write is OOP Java.
I agree with you. I was immediately struck by the tone of the article that the author was "fortunate" enough to be interviewed by Google. Maybe it's because I've been working for nearly 20 years now and am becoming an "old man", but the feeling that Google is this promised land of opportunity rubs me the wrong way. Every company has pros and cons and I suppose Google may have a lot of pros (free food, good perks, and good pay) but at the end of the day, it's just a big company where you work on large projects and will likely have little say in the final product's look and feel.
Recently interviewed at Amazon and I got that vibe as well. Didn't really "sell" the job to me at all, just assumed I would automatically be in love with them. Obviously they're there to evaluate my skills, but in my opinion an interview should be something of a two-way street.
I think it is a bigger issue of signaling. Just like going to Harvard or Stanford. Proving you are "good" enough to get a job at FAANG get you pay and RSU but then also sets you up for all future opportunities.
More money allows you to retire earlier (or have a big enough cushion to take time off as sabbatical or to start your own company or whatever), which is a huge benefit IMO.
I thought for the past five years FAANG, maybe Netflix aside, was supposed to be passé. Unicorns like AirBnB or Stripe were supposed to be the killer workplaces to strive for. Though I suppose with the potential IPO busts this year (see Uber) the dream aspirant tech workplace might be changing, too.
Are you sure you didn't sign any NDA, preventing you from working on ideas you might get during interview? Google used to have pretty water-tight policy on outflow of ideas (but inflow was encouraged) during interviews.
(Usual disclaimer: IANAL, just a person who's worked / consulted several places and signed several NDAs in his life.)
Here's the thing: the NDA you sign when you go onsite covers the company's intellectual property. It might cover the interview questions they ask you, but if they're not paying you for the work you do as part of a take-home interview question, and if you're doing it on your own hardware - it's not theirs, it's yours, and they have absolutely no say in what you do with that work afterwards.
Moreover: if you are subsequently hired by the company, they still don't own the work you did on your own time and hardware. (However: if you continue to work on it while employed by them, they might then own that follow-up work. Check your employment contract carefully, and if you're in a position to insist that employers do not own your off-hours work, insist away.)
Now, if you're currently at another company, and you complete the interview task on their hardware, and especially if you make the mistake of doing that work during company work hours...well, your current employer might then own your interview work. This is a matter usually covered by your employment contract (and another excellent reason to read those carefully).
They have nothing of the sort, as of last year. The ask politely that you not share the specific design prompts or your solutions, so that future candidates can't copy you, but they're all over medium anyway. There are no legal teeth behind the request.
This applies to the design prompt specifically, which is an early-stage take-home thing. I don't recall whether google had an NDA for the onsite interview portion itself, but I think their badge signin thing might have one baked in, and other companies of equivalent size had paper NDAs as part of checkin. Presumably they only care about you stealing the cool stuff you hear about, and you wouldn't be in danger for running with their (intentionally generic) interview prompts on your own.
An interesting aside is if the NDA is bidirectional? In most interviews, the candidate would discuss their past experiences and past projects, which could contain sensitive information via unintentional leaking. Perhaps even a side project that the candidate is working could be covered with NDA. Thoughts?
We make our candidates sign an NDA before an on-site interview, but do so because of the risk that they see/hear some of our internal information during the interview process not to entrap their ideas.
Getting hired at Google is as much of a Lottery as anything else. You really can't look at interview questions and beat yourself up. There are Google employees who got hired doing much less, and other employees that had to work even harder to get hired.
At that level, anyone who gets called to interview for Google is already qualified for the job. It really is a lottery at that point if you get the job.
My last company did interviews like this - I was always disgusted at the way most of the team would sit around and just find things to pick on in the person's code. Dumb stuff too. It became an excuse for a lot of the engineers to just make fun of someone who had successfully completed the program, but didn't take the time to JavaDoc everything and follow the same code style best practices we did. Things you can easily address with a bit of training and code review, and that maybe they already do when they're not trying to interview for a lot of jobs while still in school. Sad, really.
The way to handle this is to _write down your grading criteria ahead of time_. If it's not important enough to be front of mind when designing the challenge, it's not important enough to bring up when evaluating the output of a candidate you've really liked so far.
Not saying this approach was correct, but given a choice, what type of interview would you prefer? Hackerrank? In person, programming challenges, smaller take-home assignment, other?
As both an interviewer and interviewee I prefer in-person. Where you simply meet 1:1 with a few people - some talk to you about fit / experiences and some sit and work with you on a problem. Ideally, you would do the second in a work environment you're used to on a real computer instead of a whiteboard, and I haven't had a chance to do that before. I've seen a few poor performers slip through this before, but I do feel it gives a lot more room for people who just misinterpret what you're going for initially because there's more room for immediate feedback [1]. I've also had some disastrous experiences working with people who were amazing coders but were terrible and sitting down and talking through a problem with someone - I'd rather have a mediocre coder I can talk to. It gives the interviewee a finite time bound, and the company is spending equal manpower resources of their own (instead of giving you a 5-hour problem that you might secretly spend more on, when maybe they already are leaning towards a no but wanted to give you a final chance before ghosting). The downside here is it depends a lot on the consistency and skill of the interviewers, but I believe with good culture and training you can make most people good at this.
[1] edit: Also, I think you can tell a lot about how qualified someone is by how they clarify the problem and how they communicate their thinking - not just the code they arrive at.
I prefer a small take-home project with a follow-up meeting to go over it. The follow-up meeting to discuss the project is crucial though, as I get to asses if I think the critique of the code is reasonable and I can make more in depth comments about the project (what was good, bad, areas I might improve if I had more time, etc) not the typical thumbs up/down with no clue as to why that is typical of companies these days.
A very small-scale assignment with a work/difficulty level on the order of magnitude of an Advent of Code single day challenge seems reasonable. And a follow-up discussion afterwards is crucial to ensuring that the interviewer has put an appropriate amount of effort into actually reviewing the interviewee's code so that it wasn't all just a waste of time.
One California company that used cheap labor in third world countries to interact with customers asked me to create a scheduler so that available workers would make the system have 24 hour up time, log all employee actions, and determine employee status (fire or not) based on employee action history, they were doing it mostly manually at that point so the completed task probably would probably have fit into their production system (got access to old production data for making system), also 24 hours for a first pass, and 48 hours for a fully functioning system, for zero pay - also, they wanted someone open to writing software to replace the third world labor after this task was complete.
Great job. One should always do the best work possible. If this means someone else rejects your efforts, it'll still be a tangible satisfactory memory that, like this specific example, could turn into something more.
Some companies use a process where the candidate doesn't see the challenge until pressing "start", and then they have x hours to complete it.
This seems to be far superior for obvious reasons:
1) It normalizes the submissions, making the review process more accurate/fair
2) It removes any ambiguity or implication around the amount of time that should be spent
3) The amount of time a candidate has/is willing to devote to an unpaid homework assignment is likely not meaningfully correlated to their ultimate success in the role, but the process described by OP likely over-indexes for that.
When I was in college I preferred take-home coding challenges and whiteboard coding questions than talking through experiences.
...Because I had no experience, and school projects didn't count.
I had time that no one is paying for, I had skills (at least i believed), but no interviews because of experience. So coding challenge is the one thing that I can compete fairly with people with experiences, and I even had an edge because I value my time less.
Of course, now I don't. But I was thankful that it is a thing so I can land my first job.
This how interview task should be made : generate something joyful and useful ; instead we usually have boring throw away to do task.
By the way I had the same @ https://www.captaincontrat.com/ the task was to create a pokemon game, and from that task, I am building my current startup ;) (more on that later)
>No one acknowledges the fact that in reality, you’re about to dedicate up to 5 working days on this task, with the potential for them to ghost you straight afterwards.
Is this as common as the author makes it sound? This seems like shady practice. I've heard of marketing firms doing this as a way to get 'free' ideas/work
Someone I once worked with asked me a question that I think is relevant here and I will never forget it (roughly translated to English): Do you work to live, or live to work? It's beautiful, simple, and for me it put into perspective what really matters. I value and enjoy my job but I refuse to dedicate my life to it.
It is interesting to see this outrage from a lot of candidates. Back in "my" day (ie 10-15 years ago), it used to be that treatment of employed engineers was shoddy enough (without the benefit of decent compensation or environment) that it was relatively less risky to start a startup (aka ISV). But back then interviewing was specific, experience based and not standardized. Ofcourse as companies caught on and started selling the "we are Agile(tm) but also pay really well and pamper you", the incentives to start your own decreased. This resulted in an increased supply of candidates (at least for the "branded" companies). I wonder if the next catalyst to start startups is the shoddy treatment of candidates who are now pretty much moved through the cattle class of interviewing pipelines?
Wouldn't it be interesting if everyone participating in these types of interview challenges refused to do them unless the work could be turned into a reference piece like this guy did?
I doubt many companies would like that, but if a majority of people refused to participate without that clause, who knows? Clearly several people in this thread felt like they were being asked to do work which would be put into production and this would prevent that (though I personally am REALLY skeptical of those stories).
It seems crazy that people are being asked to do work for even a few hours for "free" much less several days of work. If you are doing that, it seems totally fair to indicate that this work will be made public afterwards and made into a portfolio piece.
This sentences stuck with me and clearly illustrates how screwed, unhuman, and selfish hiring is in this business
"No one acknowledges the fact that in reality, you’re about to dedicate up to 5 working days on this task, with the potential for them to ghost you straight afterwards."
I was ghosted by AirBNB once, and I'm fine with not being good enough, but no feedback and never a response was pretty freaking rude. You ask me to do a task, i put part of myself into do it, and then no response, ever. I'm glad they never responded, i don't want to work for a soulless company such as that.
The last take home thing I did, I put it up on GitHub with an MIT license (didn’t sign any NDAs/etc). I figured if there was a chance they would actually use it, might as well open source it for everyone.
Don't found your startup based on the first idea you fall in love with. Don't found your startup based on the first idea you fall in love with. Don't found your startup based on the first idea you fall in love with.
Make a list of attractive markets to play in, talk to a lot of insiders in those markets, choose three ideas out of at least 30 that you generated, put some serious work into validating their business models, then let yourself fall in love with one and spend 10 years sweating to make it a reality.
I like the concept. I use Strava a lot for cycling, but it's not very good for non-cardio activities. Lots of people I follow also track their strength workouts on Strava, but I haven't done it myself as I find it hard to track progress when the system is not made for it.
Before I have used Fitocracy, I'd say this may be the biggest competitor to this, as it is (at least 5 years ago when I used it) more tailored for strength, where one can log specific exercises and track them over time.
One assumes that, now that you have told the world you got the idea there, that they will show up soon to claim your startup as their intellectual property.
> I was told it was extremely close and that I should reapply in 12 months,
It seems really weird to me to put this burden on the applicant. If someone is above the hiring bar but just got edged out, shouldn't they go into the bucket of people to contact as soon as there's an opening? And for designers at a place the size of Google, wouldn't that be much sooner than 12 months?
I interviewed at Google and was told something similar. However, in their case they did reach out to me about 12 months later asking if I wanted to reapply. This was for a dev role though.
Biggest takeaway to me is that the candidate didn't do as asked because they thought they knew better. When someone asks you to do 3-5 hours of work, do that.
If it's obvious you spent DAYS on this, what does that tell me as the interviewer/recruiter? I have no idea if this person can follow instructions and I have no idea what this person can do in 3-5 hours.
A lot of people are saying they declined to continue interviewing after receiving a take-home programming assignment that was too difficult to do in the expected timeframe. I wonder if the better tactic would be to simply tell the recruiter that it's too difficult, and see what the company says.
In regards to a lot of talk about unfair pre-interview "projects", I highly recommend everyone ASK for an interview before working on the assignment, or if the assignment is short enough, work on-site, and request a code review in person.
Product looks good. I have been working on an app like this but haven't had the time to complete (only time for wire frames, research and other peoples stuff). It's good to see what should be.
This isn't typical startup origin story IP OPSEC. Aren't you supposed to say the idea came to you in the shower, the day after you and a couple coworkers quit your jobs?
I love metrics that are essentially irrelevant but sound very cool like "our BETA users have burned two million calories collectively and lifted 500,000 tonnes".
This is nice. I've been working on something like this for a while but never finished(only time to build mocks and gather research). Good to see what could be.
Hahaha. TLDR: Google asked a designer take home question. Candidate did a phenomenal job. Google still rejected candidate. Candidate turned the take home assignment into a real startup.
Just goes to say how much the industry sucks at evaluating candidates in general. This would have been a phenomenal candidate.
A lot of people don't realise it but a lot of the tasks you do for you employer could very well have been a startup. There's a reason your employer has you working on those problems: They know what their clients want.
Cisco is famous for funding startups by its fromer employees that satisfy their client wants. Google might be trying something similar with Area120?
Being successful at startups is although an entirely different matter, and I guess that's why most people don't take up the opportunity. The paychecks are golden handcuffs, for some.
> The component of entrepreneurship that really matters is domain expertise. The way to become Larry Page was to become an expert on search. And the way to become an expert on search was to be driven by genuine curiosity, not some ulterior motive.
...
> At its best, starting a startup is merely an ulterior motive for curiosity.
> ...you don't need a brilliant idea to start a startup around. The way a startup makes money is to offer people better technology than they have now. But what people have now is often so bad that it doesn't take brilliance to do better.
...
> The best odds are in niche markets. Since startups make money by offering people something better than they had before, the best opportunities are where things suck most. And it would be hard to find a place where things suck more than in corporate IT departments. You would not believe the amount of money companies spend on software, and the crap they get in return. This imbalance equals opportunity.
> If you want ideas for startups, one of the most valuable things you could do is find a middle-sized non-technology company and spend a couple weeks just watching what they do with computers. Most good hackers have no more idea of the horrors perpetrated in these places than rich Americans do of what goes on in Brazilian slums.
> Start by writing software for smaller companies, because it's easier to sell to them. It's worth so much to sell stuff to big companies that the people selling them the crap they currently use spend a lot of time and money to do it. And while you can outhack Oracle with one frontal lobe tied behind your back, you can't outsell an Oracle salesman. So if you want to win through better technology, aim at smaller customers.
sorry for being offtopic: i see you list people you follow in your profile. Are you using a 3rd party service to follow them, or using the hn api yourself ?
“My mind instantly went to an app I love, Strava. I use it almost daily ...”
So do I.
And the other day, 200m into the run, I took a first glance at Strava and noticed it was way off (gps error). Ususally I would reset and start over. But, that one time I just turned it off. 30 minutes later I was back. I don’t know how far I ran (5 point something), I don’t know how fast (also 5 pint something) and obviously have no idea how my performance on sections compares to others.
But it was a great run. A bliss.
So, I’m not saying fitness activity tracking is no good or that google interviews are. I’m just saying, that, yah, every once in a while, in both, ignorance, is, well, you know...
I recently went through an interview with a company posting on HN and was shocked by their challenge.
They provided their current UI (barebones dashboard) as well as instructions to launch the backend server locally and requested I upgrade their UI to include a side panel for filtering a data table.
The task itself wasn't difficult, but the realization that these scum bags wanted me to upgrade their current product under the guise of a challenge. I instantly knew this wouldn't be a company I'd want to work for even if they did provide an offer.
They had given me a live demo of their product on a call beforehand, so I know they didn't have this. They wanted me to design a feature they didn't have yet on their current product.
Depends on if you're happy just being a hired gun instead of an employee.
The real problem is that they've broken the social contract implicit in the interviewing process, which is that the work you're doing is only to prove yourself and that they won't benefit by it except to be able to gauge your skills. If instead they had been honest up-front that they wanted to hire you short term as a trial period type thing with the possibility of hiring you as a full employee down the line, that would be fair.
Not worthwhile, but it would have been interesting if you'd completed the project and included some telltale sign that you'd authored the code.
Then, if you found they'd deployed it without you having granted them a right to use your copyrighted work, you'd have an interesting basis for a lawsuit.
