Seems like a waste of money to me.
Training some NN is just matrix multiplications. Why would you need a custom silicon to take advantage of the software ?
I suspect a GPU is optimized for this operation all the way to software level. I can’t imagine apple will get performance benefits.
Maybe it’s cheaper or provides better access to hardware, but I don’t think performance is at play here.
Well, when you have about $200B cash sitting around, it would be silly to not invest a few B of that into foundational bets. Owning your own Silicon is the A-game of vertical integration. If you can even marginally improve performance / power ratio with deep hardware/software integration (say 10% better), compared to the industry, you can start doing things nobody in the industry can. Who knows, they may be able to flip that work into their VR stuff and other "small device" categories.
Perhaps like they did for their silicon on desktop, they are going for performance parity with a fraction of the power requirements? Power supply driving all those GPUs is currently most AI companies biggest costs IIRC
MLX on silicon rocks, now imagine tuned neural engines to transformer or mamba architectures. Yes it’s just matrix multiplication, but they can optimize the chips for their level of precision, memory bandwidth and unified memory. Not to mention the future of ai is an electric power battle, who outperforms everyone on performance per watt?
This is a wild comment. Why aren't Apple just buying Qualcomm chips for the iphone? Seems like a waste of money to build Apple Silicon.
Apple silicon is today a core part of Apple's value, it's a massive differentiator between them and the rest of the market. They also have been massively investing in custom AI processing (on the edge, not server) for ages now, they were shipping a neural engine in 2017. And that's not to even getting started on the fact that Nvidia has basically dominated the market in GPGPUs and I know lots of people aren't that into the lore of Apple but Apple has hated Nvidia for a long time. There's absolutely a market here that they can capture and if not for selling the chips into servers, but purely to drive the back end of their own AI services which they're probably going to have to deploy to a billion smart phones.
Take 10 bucks from every iPhone sold and invest it in building Apple's new LLM - that'll give you a $2.5Bn budget. It's easy to see how building out custom hardware for it could make sense.
I suspect a GPU is optimized for this operation all the way to software level. I can’t imagine apple will get performance benefits.
Maybe it’s cheaper or provides better access to hardware, but I don’t think performance is at play here.