> Google knowingly made their search results shittier and shittier for years
unfortunately this extends to youtube too. now they have a new shitty trick. you click on the link and they randomly give you a completely different video.
> It is unlikely people are going to switch en mass to open source models
It depends on the task at hands. For complex tasks no way personal computer can compete with giants data centers. But, as soon as software becomes available, users will gladly switch to local AI for personal data search / classification / summation, etc. This market is potentially huge, for private sensitive there is no other way.
Do you think the majority of humans will do this when there is a chatbot already available by default or by saying a wake word?
There is a growing constituent that doesn't even own a computer and rely on their phone only.
You have to step out of your shoes, where you are interested in these things, to consider things from a non-technical person's pov. There is a lot of unknown in having to decide what model to use... or they can just use the one put in front of them that doesn't drain their battery
I spend a lot of my time stepping outside of my shoes and making technical things accessible to non-technical people.
People having on-device private AI is something Apple is already pushing. Not sure if you've had a chance to catch up on that lately.
The important parts to explore are what people aren't doing, and what will be ready for people to adopt.
Still, this isn't the perfect example and I think it's a little unrealistic to use this as the reason to shut down the possibility of something easily existing on-device. But it is an example.
It's true, you're focusing on specifically how LM Studio works. But I provided an example that makes it a few orders of magnitude to use on a desktop, and for the interested, this is enough to learn from youtube.
You should see how non-technical people are learning to use AI and teaching others, it is very eye opening.
I would look outside of what's happening in LLM / AI to see what things have looked like historically for other technologies. Right now, anything in the AI field is early adopters and not evidence for what will happen when mass adoption happens.
Search, browser, social media, email, laptops, phones, cloud providers, word processors, business office suites... brand name vs open source... it's a telling story
I'm not against people using local models, but from personal experience, I have a hard time seeing the average consumer choosing this route. People avoid the hassle of self management/hosting
> poorly designed government intervention due to misunderstanding of the dynamics behind the process (homelessness)
Major drive is easy to understand, just cross the border and you are homeless on full support. Millions did with the help of dems, future voters. No language, no jobs, no skills. Of course they will vote for free food if they get this option. I.e. for dems, which was the whole idea.
Are you saying that the open border is unintentional and somewhere between Trump leaving and Biden taking office (probably the moment the "kids in cages" became "minor detention facilities") the border patrol suddenly became incompetent?
Only if it does nothing. In fact Google is one of the major players in LLM field. The winner is hard to predict, chip makers likely ;) Everybody jumped on bandwagon, Amazon is jumping...
I often use ChatGPT4 for technical info. It's easier then scrolling through pages whet it works. But.. the accuracy is inconsistent, to put it mildly. Sometimes it gets stuck on wrong idea.
Interesting how far LLMs can get? Looks like we are close to scale-up limit. It's technically difficult to get bigger models. The way to go probably is to add assisting sub-modules. Examples would be web search, have it already. Database of facts, similar to search. Compilers, image analyzers, etc. With this approach LLM is only responsible for generic decisions and doesn't need to be that big. No need to memorize all data. Even logic can be partially outsourced to sub-module.
It's impossible. Meta itself cannot reproduce the model. Because training is randomized and that info is lost. First samples a coming at random. Second there are often drop-out layers, they generate random pattern which exists only on GPU during training for the duration of a single sample. Nobody saves them, it would take much more than training data. If someone tries to re-train the patterns will be different, which results in different weight and divergence from the beginning. Model will converge to something completely different. With close behavior if training was stable. LLMs are stable.
So, no way to reproduce the model. This requirement for 'open source' is absurd. It cannot be reliably done even for small models due to GPU internal randomness. Only the smallest trained on CPU in single thread. Only academia will be interested.
If we can keep unlimited memory, but use only a selected relevant subset in each chat session. This should help. Of course the key is 'selected', it's another big problem. Like short memory. Probably we can make summaries from different perspectives on idle or 'sleep' time. Training into model is very expensive, can be done only from time to time. Better to add only the most important, or most used fragments. It likely impossible to do on mobile robot, sort of 'thin agent'. If done on supercomputer we can aggregate new knowledge collected by all agents. Then push new model back to them. All this is sort of engineering approach.
It's a matter of opinion how much open model should be to be called 'open source'. Looks like some believe they have the right to define it for everybody else to use. Like for software. Have to disagree. Why don't we introduce a separate term: 'open source, training infrastructure and data included for free'?
"open weight model" is confusing, because actually the architecture is open too, only data is missing.
It's a different animal. In general you cannot reproduce the model even having all the training data. There are too many random factors and nobody keeps track of them. Just pushing the training data is done at random from the dataset. This results in some interesting facts. Having the model and the data it's impossible to say if the model was trained on that exactly data. All we can say is that some pieces of that data were used in training, in some cases. Model can be 'watermarked' in hard to detect, stable to quantization and finetuning way.
So, you cannot have a reproducible, 'open source' in its strict interpretation, model.
unfortunately this extends to youtube too. now they have a new shitty trick. you click on the link and they randomly give you a completely different video.