Dude I’m literally—right now— using an LLM to fuel and *build* new features in my production website (ablf.io). I’m just some random eng. I’m sure there are many like me. And I’m not talking about gpt just recommending the odd function. It builds entire modules and writes comprehensive test suites. I’d rate its competency as similar to a junior dev. I guess I’m lying to right? It helps me learn about and write ML and NLP stuff I’m very new to. It’s basically replaced stack overflow for me.
"Dude", there is no doubt that LLMs can generate code, and there is also no doubt that they can't generate correct code reliably and in fact they are extremely inaccurate. We know this because OpenAI, DeepMind, Salesforce (I kid you not) and others have tested their systems and though the measures of performance they use are arbitrary and designed to make them look better than they are (like the n@k measure which just lets the LLM guess any number of times) they still score very badly. See the results I quote in the above comment.
So I have no doubt you are literally, right now, using some LLM to do stuff, I just have no doubt that it is not doing what you think it does.
You say you're an engineer? I know that means you write code, but the first thing that's drilled into engineers in training and in work is that you don't just make a thing and call it a day, you make sure to understand the properties of the thing you built and what it can do, and what it can't. Like you don't just put some planks on stilts and say "here's a bridge, come and drive your cars over it". You sit down and do the maths and decide what loads the bridge can take (and you optimally do this before building the bridge). So have you done anything like that? Do you have any way whatsoever to tell how often your system works and how often it shits itself?
