Unreasonable pessimism considering state of the art velocity. Determinism, while a challenge to solve for, is not entirely impossible. There is value even if you're using an LLM to code assist with integration builds (using OpenAPI specs) or breakage when an API changes suddenly.
(contributed for several years at a low code/no code prodcut org, familiar with integration maintenance and what integrating with APIs at scale looks like)
I don't think parent is being unreasonable if my perception of Gorilla is correct here but
Lot of mission critical apps cannot risk even 0.01% chance of hallucination or some blackbox process
Like I don't think a direct LLM-2-API is feasible for even enterprise clients unless LLM generates that 2-API code layer that can be audited but right away that negates the need for Gorilla
Seems like "just trust this blackbox because everybody is writing wrappers around it" is similar to "throwing redux because facebook said so" 7 years ago. Definitely seeing some parallels with technological pollyannaism with blockchain.
I agree you cannot trust unsupervised LLM output for mission critical M2M use cases, but (imho) it will help you move faster to create and maintain integrations with human supervision (code->SVN->test harness part of CICD->human review [1] and editing + proposed fixes for failure detection via Sentry [2] or similar). My apologies I did not make that more clear further up subthread.
To determine if this creates a positive value creation trajectory, some implementation and caretaking of the code generation pipeline is necessary.
doing what i do with latent.space and ai.engineer, it was nice for someone to call me pessimistic about ai for once haha. i enjoy that on HN usernames are so de-emphasized.
https://news.ycombinator.com/item?id=35631555
(contributed for several years at a low code/no code prodcut org, familiar with integration maintenance and what integrating with APIs at scale looks like)