Really, this is no different from the offline coding test, except that since the code is open source it might be useful to someone later.
I'd never do this Github interview for an open source project that was not very widely used, for the same reasons I'd never work on the company's closed source for an interview: exploitation.
In that sense, working on a toy problem is a big feature to me in an interview process, I know I'm not being cheaply exploited.
Also, other things equal, I prefer to do a code problem at interview because I know it will have helped filter out the fluff from getting into the company I'm applying to.
That's why it should be a cool, fun project. It's hard for me to see how one might get exploited by contributing as much time as he wants to an open-source project. Remember, the prospective employee uses the same experience to gauge what it's like to work with his prospective employer.
Assuming this company has fun work to do (you'd hope so), there's nothing stopping them just giving you their work.
It doesn't matter that it's open source if you're not going to use it and no-one else apart from them is (a very real possibility).
In essence, I'd happily agree with the article after changing this:
> "You come up with a cool idea of an open-source project"
with this:
> "You find a _popular_ open-source project"
Now you can be certain (not just optimistic) that the wider community will benefit from your work.
(edit: I'm thinking along the lines of jQuery, backbone.js, nodejs, django, memcached, sqlite, Nginx, Hadoop, Linux, ..., not some dev's/company's lonely github project)
It's true that you'll have more impact by contributing to a well-known project, but on the other hand the range of activities you can take part in is limited. Well-established projects usually don't call for high-level design activities - the sort that a tech co-founder would need to do. They're more about contributing patches and perhaps a new feature (after you've learned the project, which is usually quite big). Ultimately I think you get the same amount of karma if you work on a pet project and not a well-established one. And there's absolutely no reason why the company's sandbox project can't fill a real need and become popular with time.
Yeah, this is a non-issue. The safest way to do it is to make them contractors for the two-week period. Nothing looks bad about a two-week consulting contract.
Some trickier issues with trial periods are:
- If they currently have a job, they will typically need to quit the other job before they can do any work for you, even if it is on nights and weekends, and even if it is unpaid. Otherwise there are serious legal issues around IP. So it's no good for poaching.
- A good candidate will have several offers. Even for a two-week trial period, they will probably need to turn down those other offers. That sucks for the candidate because it will really muddy the water with those other companies if in two weeks they want a job again. They could always get offers from other companies, but if the candidate actually wanted to work at those other companies, why didn't she interview there the first time around?
Both of these are surmountable if you are a genuinely desirable employer. For one thing you have the flexibility to talk only to awesome candidates, and awesome candidates are less likely to have problems accepting offers that they previously rejected. (Perhaps even on better terms -- it's very powerful to walk away from a negotiation while you have the upper hand.)
In some countries this information turns up when you do your taxes for the end of the year. And not all job markets are as huge and "anonymous" as the U.S. job market - some (like Israel) are relatively small (a few 10ks of programmers probably) and word gets around. Besides, if you get asked on an interview about what you've done in the recent past in terms of looking for work you're basically lying if you're not mentioning a brief period of employment. Not a huge deal, but still it's not honest and some people would rather not.
Background checks will show where you've worked, unless you were getting paid illegally. Still, I agree with you. It isn't something to scream out loud.
Background checks are being restricted in CA (by laws that became active in Jan '12), but even before that, I doubt it. Background checks are more like credit checks than employment history checks, except they include law enforcement activity.
Background checks are nowhere near that accurate, they usually rely on information you, yourself have put out there, they don't have some sort of direct pipeline to your w2 forms.
"Not all developers have existing online projects they can point to when you're interviewing them."
After 15+ years of doing open source and being in Sr. Software Engineer positions where I've been responsible for interviewing hundreds of people, I'm convinced now that I would never hire anyone who doesn't have some sort of public online project. I don't care what it is, you must have some code to show.
If I were being interviewed by you I'd have a question: "If you hire me, can you guarantee that either (1) I'll be contributing to open source as part of the job, or (2) the job will leave me with enough free time and energy that I will have both the time and desire to do programming outside of work on open source?".
If your answer was "no", then my follow up question would be "so basically you wouldn't hire someone whose prior career had been spent at companies like yours?".
If your answer to the first question was "yes, option (2)", I'd have some follow up questions probing why the job is not able to provide enough interesting and challenging work to fully engage me.
I think you are missing the point. This isn't about the job you are going to be doing, this is about getting me to even consider you for a job.
To answer your questions though:
(1) I'd expect you to continue doing open source, as both part of the job and on your own. I do it, so should you. Case in point, we just integrated our site with WePay. WePay didn't have Java API, so I wrote one (along with my business partner) and we made it open source [a]. This benefited my company, open source and WePay all in one shot. This isn't the only one, I can point at many more large projects [b], [c], [d] (for just a short list) that we've either open sourced ourselves or contributed to. I have a clear history of coding in multiple languages. Anyone who is going to hire me knows my skill set without having to give me silly brain teasers or even read a resume. Why would anyone want to hire someone without a clear record like that?
(2) How you spend your time is up to you. There is a certain amount of work that needs to get done as part of the job and I'd expect that to get done. As long as you are getting your work done, in a reasonable time and of a good quality, I'm fine with whatever you do with your time. If that 'policy' doesn't leave you with enough energy, then I'd suggest either learning how to do better time management or find another profession / company.
As I already said, I wouldn't hire someone without seeing their source code. The single best way to really see it is to see how they've contributed and worked with open source projects. Bonus points if you find a unique project to create yourself.
You are saying that you only have code which is entirely private. This is most likely code you wrote for your employeer, which you can't show me anyway because chances are that it is NDA private. Or, you have code which you wrote for say, an iPhone app which you are probably never going to show to anyone. That is all fine.
However, you are also telling me that you've never integrated any open source projects into your code or relied on any non-commercial external libraries. Or, maybe you have done that.
But then I have to wonder why you've never contributed back to any of these projects that you've depended on. If you are a great dev, I'm sure you found them lacking in some way and wanted to contribute a bit back to them.
Thus, we've come full circle. Any developer that I would want to hire, has figured out that there is a massive ecosystem of open source projects out there. They are integrating them and contributing something back to them. They are drawn to these projects like flys to sh*t. These projects teach them new tricks because they are studying others code. They are drawn to give back to these projects because they are taking so much good from them. That is what makes a good developer in my eyes.
Here is a really good case in point example. Last night I was working with the amazing Underscore.js library, which I'm fairly new to. I found that I wanted to do something with it, couldn't figure it out and made a suggestion on how to improve it [1]. Someone from the community responded quickly, we had a bit of really positive back and forth and in the end I learned something new about the library. I expect that anyone I hire has the gumption to also be doing similar work.
You'd miss a lot of really good people with that standard. I used to work at Amazon, so I'll use them as an example. (Keep in mind, it's been about 1.5 years since I left, so some of this may have changed.) Amazon has a policy that requires you to get open source contributions approved by legal. That is, legal has to review every commit you want to put into open source to make sure 'it's not in a competing product', reviews can take weeks, and when you ask a lawyer every product is competing.
The whole company is built off of Open Source, but there are teams the fork the projects into internal repositories (Perforce when I left) and put a bunch of Amazon-specific 'things' into them. As a developer there, I didn't use the Open Source sites, docs, or mailing lists at all -- I used Amazon's. We had wiki's, ticket queues, and mailing lists -- all internal to Amazon. Maybe the core team that's doing the forking and merging from the trunk would or does contribute back, but for any given project >99% of the developers at Amazon don't ever work on that codebase -- or with the codebase of any of these projects.
As a developer, we just have a 'team' somewhere that does. If something doesn't work in Tomcat, we don't go find the Apache page and file a bug, or jump on the mailing group, we file a trouble ticket with the team that owns Tomcat at Amazon, and they fix it. Maybe they go to the Apache pages and mailing list, or maybe they just see the line of code in the codebase and fix it, I have no idea.
I've been waiting on approval from legal for a diff viewing tool. It has been under review for over a year now. On the one year anniversary of the ticket I pasted in an ascii art birthday cake and there has finally been some progress. It is among the many reasons amazon loses great devs.
I'm ok with missing those people, they already have a nice stable big company job they are happy at. There is a lot of value in that and I'd never want to take that away from anyone.
In high school, we were all told that in order to get into college, you needed to have extracurricular activities. I consider spending some time at night or on the weekends on an open source project part of the deal of being a software engineer. Not everything needs to be taught or done on the job. The people I want to hire make their own time to learn and grow.
It's different for big companies (Google, Microsoft etc.) that have to hire by truckloads, but small, startup companies (say, <40 people), only need to hire a couple of good people and therefore can be selective and therefore can use a filter that rejects "majority of people" (especially given that filter also strongly correlates with "people they wouldn't want to hire anyway").
Now, I'm not saying it's your problem, as you also need to be hired by only one of many companies so it's perfectly valid to ignore companies that have a strong preference for open-source contributions.
However, if you do want to be considered by such companies, it's a small investment to build a portfolio of open-source code on github.
I believe this is becoming more and more important because more and more startups are founded by people who grew up in a world where open-source projects and github are a norm, not some extraordinary event as it used to be just a few years ago.
Out of all the devs I respect and admire as mentors, the majority of them do have publicly contributed code and are involved in some open source project. Not to say there are not a few who don't, but I think the point is that the majority of really talented developers out there seem to have something.
I solved a test problem for a company once (gt 500 LOC), but it was after a pre-screen and the stated salary was 130-140k base, not including generous bonuses, in the Midwest. It was also when I had a lot of downtime, so I didn't mind.
Now, I have a hard time finding enough time for my own projects, much less for contributing something to a company just for the chance to interview.
Since you're offering single digit equity, it might be worthwhile, but for the vast majority of companies, it would warrant a pass.
How is this any better than just looking for candidates who have created or contributed to other open source projects? I suppose there would be some good candidates that came from bigger companies and maybe don't have much of a GitHub portfolio, but surely the vast majority of people that would be considered by tech startups would already have some work online.
It's all about context. While a candidate might have some code online, you save a lot of time by having some code that you're more familiar with and can utilize to see how they think about a problem.
Also it's good to see how someone reacts to digging around in an unfamiliar code base and see how they familiarize themselves.
Context is true. It's totally different to see someone work _with you_ on a project than to rummage around their commits to project you don't really know. The point is to see how it is to collaborate with someone, and that has to be a first-person experience (or take the word of someone you really value, but that's not what you get form random GitHub projects). And also there are tons of developers who never really contributed much to open source. e.g. in the Microsoft world this is not so rare, and you'll miss these people if you _require_ a GitHub reference...
The idea is good, but if you are explicitly interviewing I would add an honorarium to the whole thing.
This way it's not officially a probation - the candidate can even say they were freelancing during this time - so the CV problem doesn't come up AND you don't look the tiniest bit like you are exploiting people.
I've had some success by asking a late-stage candidate to take a day off of work, then to come and pair program for a half-day with each of two developers. I paid them the appropriate consulting day rate.
writing code during an interview is great, but almost a 180 degree experience to writing code in real life. there are a couple of factors:
* psych pressure
* unrealistic timeline
* googling is usually not allowed, whereas EVERYBODY uses it writing code in real life
* editor/environment is usually not "yours"
* etc..
hence the _way_ you choose to conduct a code test will, in most cases, matter a lot less than the actual _problem_ you choose that needs to be solved. the more applicable this problem is to what your business does every day the better. e.g. sometimes a "red-black tree" is crucial, other times things like a simple mobile CRUD app with a REST web service OR a thin, high throughput TCP/IP based protocol could be a lot more relevant.
of course you can't realistically expect a 100% bug free or even complete code from candidates, especially if you ask for things like an "app" or a new "protocol" => what really matters is _how_ people "get to it".
github or not, I hired several exceptional people not that long ago using a piece of paper and a white board. If it is not a face to face interview, skype and/or google docs are also effective.
for what its worth, at my last company once a candidate passed phone screening we would tell them that the in-person interview would include a one-hour coding assignment, and that they should tell us their preferred language and (free) editor/ide and we'd set it up. we also provided full access to google, and the tasks were reasonably easy (e.g. print out numbers in a square spiral, implement a duplicate file finder, that sort of thing). a depressingly vast majority still couldn't do it; contrariwise the good ones would have just as happily done it all on a whiteboard.
I've been mulling over the idea of writing a small buggy program, one that doesn't compile, and providing some test cases, then post it on GitHub as a first pass filter.
It achieves several aims:
1. This approach scales. Send a prospective employee the github url and they can get started.
2. The programming puzzle can be solved in about 15 minutes. Kind of like a better Fizz Buzz.
3. The interviewee doesn't feel insulted that the employer expected them to do some work for free.
4. The employer has some assurance the person will have some source control competency, to be able to check out from a Git. repository.
5. You could test the prospective employee based on the skills you require. For example: You could even have the puzzle interact with a web service. Troubleshooting the program will involve firing up a proxy or packet sniffer to figure out why things didn't work. It might be a jQuery bug. etc.
Why go through all this complication? Use bugs that have actually occurred in your app, repurpose those into coding problems, and see how the prospect solves it. You might be surprised that they fix the bug better than your existing employees (or you).
Well you don't give them the entire codebase, you package the bug-that-was-fixed into a tiny example app. I think it probably goes without saying to exclude changes in business rules.
Is this not closer to the actual job than Pascal's Triangle?
That is a sensible idea - but i would go one step further.
Ideally you want to learn about a potential candidate's design/coding/debugging/bug-fixing/testing skills. Github project collaboration over a week or so would go a long way towards that.
But you also want to see how well the potential employee communicates with other members of the team, on a day-to-day basis. Some Developers, even good developers, have strong opinions that might rub other employees the wrong way. The key is to look at how well they work as a team.
Having them spend a day or so onsite with your existing team is invaluable. You might also want to consider paying them a nominal amount for the amount of time they would spend on the project. That just adds a professional touch to the whole experience.
If I'm onsite with your team working on your code-base, someone damn well better be paying me for it. Not to mention those kinds of shenanigans are downright illegal in some parts of the world...
This is what contracting is for, no one in their right mind with any sort of real experience is going to spend 2 days on-site working for someone for free.
Seing code is the way to go in job interviews, and in my opinion can't be compared to solve puzzles, etc.
Thanks to Open Source right now there are tons of public code from people, that's why we have created masterbranch.com a service to aggregate, analyze, and list all you OS but also with the option of include non-open source code (the code is not visible if the project is not open source).
Also, the idea of open sourcing a component, a side project, etc. and ask the candidates to contribute I think is a great way to have an idea about how they work.
Thanks for your feedback. I'm going to add it to our backlog.
- About the bitbucket projects, maybe the crawler didn't get all the links, you can send us projects to add going to "Missing projects? Add a project"
- About the link to go to your profile, there is a tab named
"go to your profile" on the main nav-bar
- The progress bar needs explanations and also some work because is not working properly right now.
- The chart is ordered by the number of files. Probably we should order it using the number of files, commits, and the time but right now this's the order and maybe this is why is confusing to you.
I'd never do this Github interview for an open source project that was not very widely used, for the same reasons I'd never work on the company's closed source for an interview: exploitation.
In that sense, working on a toy problem is a big feature to me in an interview process, I know I'm not being cheaply exploited.
Also, other things equal, I prefer to do a code problem at interview because I know it will have helped filter out the fluff from getting into the company I'm applying to.