I think you are looking at a very narrow use case and deciding that because they do not make a system you'd be happy with for your niche use that they are DOA. Someone selling just under 6.5 million units of anything seems like the opposite of dead to me. Are there vendors selling more? Of course, but there are also vendors selling less. Not every Mac user cares about AI and training or fine tuning a local LLM.
Very true, my needs are niche for sure. But I’m more thinking about the near future. AI/LLMs are going to have some general applications that users are going to want, and will become the norm, and I think it’s clear that will shake out soon. Apple is at risk of being left behind because the only people working on that stuff for Apple, work at Apple. Hobbyists and researchers are on Linux/Windows for the most part. Software development doesn’t have such a large platform difference, lots of developers use macOS. But ML is different and I think they should care.
> But ML is different and I think they should care.
It’s totally this time I promise, just like, one more ~~lane~~ model.
I’m sure they do care. I wouldn’t be surprised if they land significant support for on-app processing of models, they’ve already got the chip, dropping in local models is a sensible next step, and if close to zero effort for them.
> LLMs are going to have some general applications that users are going to want, and will become the norm
I have yet to see anyone, in my personal or professional circles, use any LLM:
- for more than a week
- for anything more than cutesy trivial things.
I’m sure there’s people around stapling models into their toaster, but this is so far from the normal.
Part of Apple's problem is that they're expected to vendor support for third-party stuff. Who accelerates Pytorch or ONNX for Apple silicon, if not Apple?
They've done an okay job of that so far, but their flagship library is diverging pretty far from industry demand. At best, CoreML is a slightly funkier Tensorflow - at worst, it's a single-platform model cemetery. No matter what road they take, they have to keep investing in upstream support if they want Nvidia to feel the heat. Otherwise, it's CUDA vs CoreML which is an unwinnable fight when you're selling to datacenter customers.
I think it's possible for Apple to make everyone happy here by reducing hostilities and dedicating good work where it matters. Generally though, it feels like they're wasting resources trying to compete with Nvidia and retread the Open Source work of 3 different companies.
> an unwinnable fight when you're selling to datacenter customers.
didn't Apple pretty much throw in the towel in this market simply by choice of form factor for their computers? The sheer desperation of their users wanting a device in this space is shown in the "creative" ways to mount their offerings in a rack.
all of the user friendly things they've done by shrinking the footprint, making them silent, etc are all things a data center does not care about. make it loud with fans to keep things cool so they can run at full load 24/7 without fear of melting down.
so from that lead alone, we can make the next assumption in that Apple doesn't care about vs CUDA. as long as they can show a chart in an over produced hype video for a new hardware announcement that has "arrows go up" as a theme, that is ALL they care about.
I mostly agree, which is why I question their strategy of even "competing" at all. The existence of CoreML feels strictly obligatory, as if the existence of Pytorch and Tensorflow spurned FOMO from the C-suites. It's not terrible, but it's also pretty pointless when the competing libraries do more things, faster.
Users, developers, and probably Apple too would benefit from just using the prior art. I'd go as far as to argue Apple can't thread the AI needle without embracing community contributions. The field simply moves too fast to ship "AI Siri" and call it a day.
> The sheer desperation of their users wanting a device in this space is shown in the "creative" ways to mount their offerings in a rack.
Well you and I both know that nobody is doing that to beat AWS on hosting costs. It's a novelty, and the utility beyond that is restricted to the few processes that require MacOS in some arbitrary way. If we're being honest with ourselves, any muppet with a power drill and enough 1U rails can rackmount a Mac Mini.
>I mostly agree, which is why I question their strategy of even "competing" at all.
If it makes their camera "smarter", it's a win. If they can make Siri do something more than "start a timer", then it's a win. If they can have images translate text more accurately, it's a win. There's a lot of things that an on device AI could help users without having to do all of the power hungry creation of a model or the fine tuning. They can do that in the mothership, and just push models on their device.
Not everyone needs to do AI the way you are trying to do it
I think that's a mistaken way of viewing it. Apple's failure in the gaming space is entirely a matter of policy; you look over at the Steam Deck and Valve is running Microsoft IP without paying for their runtime. Some people really do get their cake and eat it too.
Any of the aforementioned libraries could make their camera smarter or improve Siri/OCR marginally. The fact that Apple wasted their time reinventing the wheel is what bothers me, they're making a mistake by assuming that their internal library will inherently appeal to developers and compete with the SOTA.
The reason why I criticize them is because I legitimately believe Apple is one of the few companies capable of shipping hardware that competes with Nvidia. Apple is their only competitor at TSMC, it's entirely a battle of engineering wits between the two of them right now. Apple is going nowhere fast with CoreML and Accelerate framework, but they could absolutely cut Nvidia off at the pass by investing in the unified infrastructure that Intel and AMD refuse to. It also wouldn't harm the customer experience, leverages third-party contributions to advance their progress, and frees up resources to work on more important things. Just sayin'.
