The problem I have with it is that for this to be a reasonably effective strategy you should change the arbitrary metric every few years because otherwise it is likely to be hacked and has the potential to turn into a negative signal rather than positive. Essentially your false positives can dominate by "studying to the test" rather than "studying".
I'd say the same is true for college admissions too... because let's be honest, I highly doubt a randomly selected high school student is going to be significantly more or less successful than the current process. I'd imagine the simple act of applying is a strong enough natural filter to make this hypothesis much stronger (in practice, but see my prior argument)
People (and machines) are just fucking good at metric hacking. We're all familiar with Goodhart's Law, right?
I think (but cannot prove) that along the way, it was decided to explicitly measure ability to 'study to the test'. My theory goes that certain trendsetting companies decided that ability to 'grind at arbitrary technical thing' measures on-job adaptability. And then many other companies followed suit as a cargo cult thing.
If it were otherwise, and those trendsetting companies actually believed LeetCode tested programming ability, then why isn't LeetCode used in ongoing employee evaluation? Surely the skill of programming ability a) varies over an employee's tenure at a firm and b) is a strong predictor of employee impact over the near term. So I surmise that such companies don't believe this, and that therefore LeetCode serves some other purpose, in some semi-deliberate way.
I do code interviews because most candidates cannot declare a class or variable in a programming language of their choice.
I give a very basic business problem with no connection to any useful algorithm, and explicitly state that there are no gotchyas: we know all inputs and here’s what they are.
Almost everyone fails this interview, because somehow there are a lot of smooth tech talkers who couldn’t program to save their lives.
I think I have a much lazier explanation. Leet code style questions were a good way to test expertise in the past. But the same time everyone starts to follow suit the test becomes ineffective. What's the saying? When everyone is talking about a stock, it's time to sell. Same thing.
> If it were otherwise, and those trendsetting companies actually believed LeetCode tested programming ability, then why isn't LeetCode used in ongoing employee evaluation?
Probably recent job performance is a stronger predictor of near future job performance.
so having done interviews, just because the latter may be more present, does not mean the hordes of people just throwing a spaghetti made-up-resume at the wall have gone away. our industry has a great strength in that you don't need official credentialing to show that you can do something. at the same time, it is hard to verify what people are saying in their resumes, they might be lying in the worst case but sometimes they legitimately think they are at the level they are interviewing for. it was bad before the interest rate hikes, i cannot imagine what the situation is like now that hiring has significantly slowed and a lot more people are fighting for fewer jobs.
i did interviews for senior engineer and had people fail to find the second biggest number in a list, in a programming language of their own choosing. it had a depressingly high failure rate.
Ten years of experience at one of those places where every keystroke outside powerpoint is offshored. Why would they know how to sum ints? Some people do start their careers as what could best be described as software architecture assistants. They never touched a brick in their lives, to go with the architecture image.
I have junior and senior students that struggle with fizzbuzz... But damn, are they not allowed to even do a lazy inefficient `sort(mylist)[-2]` if they forgot about for loops? That's the most efficient in terms of number of characters, right haha
But I still think you can reasonably weed these people out without these whiteboard problems. For exactly the same reasons engineers and scientists can. And let's be honest, for the most part, your resume should really be GitHub. I know so much more about a person by walking through their GitHub than by their resume.
Using GitHub is discriminatory against people who don’t code on the weekends outside of their jobs, and most people’s job related code would be under NDA and not postable on Github.
To be a capital E Engineer you have to pass a licensing exam. This filter obviously is not going to catch everything but it does raise the bar a little bit.
—-
As far as the root question goes, they are allowed to propose that, and then i can try and tease out of them why they think that is the best and if something is better. But you would be surprised at the creative ways people manage to not iterate through a full loop once.
You're right. But a lot of people that are good coders code for fun. But you're also right that not all those people push their code into public repositories. The same is true for mechanical engineers. They're almost always makers. Fixing stuff at home or doing projects for fun. Not always, but there's a strong correlation.
But getting people to explain projects they did and challenges they faced can still be done. We do it with people who have worked on classified stuff all the time. If you're an expert it's hard for people to bullshit you about your domain expertise. Leet code is no different. It doesn't test if you really know the stuff, it tests how well you can memorize and do work that is marginally beneficial in order to make your boss happy. Maybe that's what you want. But it won't get you the best engineers.
Leet code, in the interviews that I do, is not the only thing I do.
But when I am asked to do a one hour part of an interview for four interview loops a week, all the preps and debriefings, and also do all my normal day-to-day deliverables, we need some fast filters for the obvious bullshitters. The interviewing volume is very high and there is a lot of noise.
“Codes on their spare time” is not part of the job description, but “codes at all” is.
There are plenty of reasons not to code on spare time. If anything the people who are most likely to do that are often also the people who coding interviews are supposed to be privileging, fresh single college grads.
I don’t know how people would square the statements “take-home assignments are unpaid labor and unfair to people with time commitments” and then do a 180 and say “people should have an up-to-date fresh github that they work on in their spare time.”
If it would take the candidate "spending every waking moment of their lives coding" to have one or two small coding projects after a half decade plus in the field, that's a signal.
If you went to college but never made anything, that's a signal.
If you didn't go to college, and never made anything, just shut up and make something.
In a half decade plus some people pick up other commitments that are not side projects, like a pet, a child, sports, hiking, etc.
At the end of the day, it isn’t really relevant to the employer what is done in spare time off the job when they get hired, so it’s not like I should privilege their spare time projects over people who don’t do that, particularly if people don’t want to code off the clock. There are plenty of good engineers who do not code off the clock, and there are plenty of bad engineers who do.
Also, more often than not, coding off the clock for the sake of having a portfolio is not really indicative of anything. There aren’t, for example, review processes or anything like that in a one person side project, and unless I spend significantly more hours background checking who’s to say the side project is not plagiarized? People already lie on their resumes today.
In the time you took writing this comment you could've gotten the repo created and the title and description filled out. Writing text in a public readme.md would serve you better than sending it to me.
I'm not saying it should be mandatory, but they would have to show mastery some other way. Whiteboard? Live coding? Project?
I think a side project opens up the opportunity to skip that for a project presentation. This is a lot more in line with real life as you would typically code and later present that work to others. You would defend it to some degree, why you made choice A vs choice B. If you created it, you'll be able to do that.
Doesn't need to be a huge thing. Just show you can do anything at all really at the junior level. Intermediates can show mastery in a framework with something with slightly more complexity than a "hello world".
Typically the people without side projects also make excuses to not do those either.
If I had a company I'd offer looking over an existing project or a project where you create a side project of your choice without any further direction.
So not mandatory but the easiest way to go probably. Once you apply to my company you'll have one to show for next time at least.
(If you want to write the project out on the whiteboard instead I guess feel free, that seems hard though.)
Many people do not have side projects. Few people working as software engineers were not tested in some way.
I think it's more useful and more fair to give candidates some direction when I request something. What scope is enough? What tests are enough? We define side project differently or you would expect much more time from candidates than I do.
I used to think so. But real tasks have acceptance criteria. Seeing how candidates work with loose criteria has told me more than telling them in effect to read my mind.
The problem I have with it is that for this to be a reasonably effective strategy you should change the arbitrary metric every few years because otherwise it is likely to be hacked and has the potential to turn into a negative signal rather than positive. Essentially your false positives can dominate by "studying to the test" rather than "studying".
I'd say the same is true for college admissions too... because let's be honest, I highly doubt a randomly selected high school student is going to be significantly more or less successful than the current process. I'd imagine the simple act of applying is a strong enough natural filter to make this hypothesis much stronger (in practice, but see my prior argument)
People (and machines) are just fucking good at metric hacking. We're all familiar with Goodhart's Law, right?