So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the real problems that aren't just gluing some libraries together your job is safe.
My dude, putting aside the fact that leetcode problem solving isn't factoring in the majority of programming use cases... you do realize this is better than a lot of professional programmers right ?
It's truly amazing seeing the posts shift in real time.
AI will overperform at questions whose answers are in its training set. These leetcode questions has lots of answers posted online, any human with internet access can just google those and copy paste the solution and get a better score than these language models.
This effect is stronger the larger the model, so likely most of the improvements here is that the model has better memory of the solutions that are already out there.
What this tells me is that doing technical interviews online is going to become even more of a hassle.
Either you'll have to completely surrender yourself to a battery of various "anti-cheating" software, or the problem difficulty is going to go through the roof.
What is "Leetcode (hard)" specifically? Searching for it brought me to a website (https://leetcode.com/) but I'm assuming that is referring to some special list of problems that are meant to be harder than others, but I cannot find that.
people are impressed because you never need to do leetcode hard problems in any business anywhere in your career to make a few billion dollars. I would fire anybody using company time trying to reinvent some academic search thing instead of just using the hashmap.
Reimplementing quicksort in a language the rest of the company doesn't use in a part of the software that no one ever touches is fireable, sure, but I suspect that on the way to ChatGPT there were more than a couple "Leedcode hard"-grade problems that OpenAI engineers and PhDs solved because the library for that didn't exist before.
So it's better than GPT-3.5, but still pretty pathetic at hard Leetcode problems. If your programming job is closer to leetcode easy problems you might be in trouble, but for the real problems that aren't just gluing some libraries together your job is safe.