Why does an LLM have to be better than you to be useful to you?
Personally, I use them for the things they can do, and for the things they can't, I just don't, exactly as I would for any other tool.
People assuming they can do more than they are actually capable of is a problem (compounded by our tendency to attribute intelligence to entities with eloquent language, which might be more of a surface level thing than we used to believe), but that's literally been one for as long as we had proverbial hammers and nails.
> Why does an LLM have to be better than you to be useful to you?
If
((time to craft the prompt) + (time required to fix LLM output)) ~ (time to achieve the task on my own)
it's not hard to see that working on my own is a very attractive proposition. It drives down complexity, does not require me to acquire new skills (i.e., prompt engineering), does not require me to provide data to a third party nor to set up an expensive rig to run a model locally, etc.
Then they might indeed not be the right tool for what you're trying to do.
I'm just a little bit tired of sweeping generalizations like "LLMs are completely broken". You can easily use them as a tool part of a process that then ends up being broken (because it's the wrong tool!), yet that doesn't disqualify them for all tool use.
Yeah but the sampling process required to determine what they are good at is not free either. (For starters, it's consuming huge amounts of public research funding and compute, but let's not go down that rabbit-hole)
If you can't find a use for the best LLMs it is 100% a skill issue. IF the only way you can think to use them is re-factoring complex java codebases you're ngmi.
So far I haven't found one that does my dishes and laundry. I really wish I knew how to properly use them.
My point being: Why would anyone have to find a use for a new tool? Why wouldn't "it doesn't help me with what I'm trying to do" be an acceptable answer in many cases?
Well I found the exact reverse: people saying LLM are useful actually need them to be useful, so they could boast about their "prompt engineering" "skill" (i.e. typing a sentence) and "AI knowledge". I've seen a caricature of this a few hours ago on LinkedIn, from a "data guy" saying devs not using AI are gonna replaced by those who do. Yet it was very clear from his reply to comments he never wrote code and wasn't a position to give an opinion on the matter, especially one like the extreme and rude one he wrote.
Both your and GPs observations (and many more) can be true simultaneously.
Some people are quick to dismiss any new technology as useless; others are quick to hail it as the thing that will take everyone's jobs in just a few months (and might consider that a good or bad thing) or solve any number of humanity's hard problems.
Usually one or the other will be seen as slightly more accurate in retrospect, but since both ultimately come from a knee-jerk reaction to something new, with rationalizations bolted on to support their respective case, most of these can be safely ignored.
Remove all the parroting of technical jargon, wishful thinking, appeals to morality etc. and you essentially have two crowds arguing why this time the roulette ball will surely fall on red/black (and everybody forgetting about the zero/green).
Copilot fails the cleanly refactor complex Java methods in a way that I’m better of writing that stuff by my own as I have to understand it anyways.
And the news that they don’t scale as predicted is too bad compared to how weak they currently perform…