The only way around this is to sit with the candidate and carefully review the code they submit. Ask about design decisions and navigate the design space together (How would things change if memory were most important? What if input were 1e6 times bigger?)
You could also show them another solution (maybe one with a bug?) and ask them to explain it and compare with their solution.
If they are leaning heavily on an LLM it should show up during the follow-on discussion.
> You could also show them another solution (maybe one with a bug?) and ask them to explain it and compare with their solution.
IMO guided reviews seem like a great option in general. You have complete control over what goes into the code to review and it wouldn't even have to be code in the first place. Could also be documentation, processes or architectures.
And that skill remains valueable even when LLMs become the most effective way to write code at some point.
But to be honest, I don't think LLMs will ever replace "coders". Language theory has always been an important part of CS degrees, and that's because in programming you need an efficient and precise way to express what exactly you want the computer to do.
In that way I don't see why whatever we end up feeding into LLMs shouldn't be considered "code". I guess that in the end we'll get smaller LLMs put into compilers to enable better optimizations.
What are you trying to accomplish with these take home tests to begin with? If they can successfully solve it with ChatGPT that implies they understand how to use tools which are paramount to developer success. Do you want them to waste a bunch of their time solving a pointless problem? Is that how you treat your employees, too?
Sorry it wasn’t meant to come off as an accusation, more of a thought experiment: what would prove they would be successful in the job: proving they can solve a trivial problem in thirty minutes? I assume your team is not only solving the same trivial problems over and over again.
I do have an idea I suppose. Present them with a problem and a solution and have them lead a discussion of the merits of the solution, what alternatives they might propose and so on. There’s some issues to avoid with this approach tho, like presenting a pair with a correct response in mind and then judging people against that standard rather than the standard of “how are they thinking through this problem, the drawbacks and benefits, are they presenting their ideas effectively and respectfully, etc”
It also primes them to expect a certain sort of environment coming into the job, participating in meetings and collaborative solutions instead of just being a code cowboy. Adapt your specific discussion to how you expect them to conduct themselves, probably present your company collaboration guidelines and meeting CoC as a precursor to the meeting, idk I’m just spitballing here. I’ve struggled with this problem a lot hiring people.
You can tell in an interview if they are focused on the interview or the job. If there is never any talk about the job, if it's all about some insanely drawn out interview process you know that's all they care about.
I've seen startups (even some here on YC) who have been hiring the same roles for literally years, they don't know anything else.
They will complain in Tweeter and the mob will get angry. This makes more sense if the worker moved from another city or resinated from a previous job.
Not really. Glassdoor is more about being "squeaky clean" than being "transparent", they'll delete any review that goes against their many arbitrary guidelines. As for Twitter sure, but you get shadowbanned there for all kinds of things - including not paying for the check.
You could also show them another solution (maybe one with a bug?) and ask them to explain it and compare with their solution.
If they are leaning heavily on an LLM it should show up during the follow-on discussion.