> Why does an LLM have to be better than you to be useful to you?
If
((time to craft the prompt) + (time required to fix LLM output)) ~ (time to achieve the task on my own)
it's not hard to see that working on my own is a very attractive proposition. It drives down complexity, does not require me to acquire new skills (i.e., prompt engineering), does not require me to provide data to a third party nor to set up an expensive rig to run a model locally, etc.
Then they might indeed not be the right tool for what you're trying to do.
I'm just a little bit tired of sweeping generalizations like "LLMs are completely broken". You can easily use them as a tool part of a process that then ends up being broken (because it's the wrong tool!), yet that doesn't disqualify them for all tool use.
Yeah but the sampling process required to determine what they are good at is not free either. (For starters, it's consuming huge amounts of public research funding and compute, but let's not go down that rabbit-hole)
If
((time to craft the prompt) + (time required to fix LLM output)) ~ (time to achieve the task on my own)
it's not hard to see that working on my own is a very attractive proposition. It drives down complexity, does not require me to acquire new skills (i.e., prompt engineering), does not require me to provide data to a third party nor to set up an expensive rig to run a model locally, etc.