Agreed. After several rounds of setting up various python environments and tinkering with directory structures and debugging glitches and quantizing models just to end up playing around for a few minutes and getting bored, it would be nice to have the experience just be seamless. I wouldn't try to set up a workflow around seriously using what's out there to run on localhost now.
That said, non-subscription is essential, and that's probably going to be a heavy lift considering how quickly things are evolving.
I've been trying to push things in that direction with my LLM tool - the idea is to have Python plugins which you can install that do all of the irritating details to get a model setup.
I've not yet been able to solve the challenge of needing CUDA etc for some models though!
Cool! I've followed your instructions and your blog quite a bit as I've experimented with running local LLMs as well as stable diffusion. It's been especially helpful, as python is not my language or usual environment. Your patience at hacking your way through each new iteration and presenting what's important about them is astonishing; I personally think I'd have gone mad, but you've done great work in charting the territory.
That said, non-subscription is essential, and that's probably going to be a heavy lift considering how quickly things are evolving.