What's your opinion on the coming (or not) GPU crunch?
When looking at cloud GPU availability and current trends (no one except some enthusiasts and bigtech is finetuning and serving on a large scale yet and results keep getting better and better), I fear we will run into a situation where GPUs will be extremely expensive and hard to come by until supply catches up?
I ordered a high end PC with 4090 for the first time in years (normally would always prefer cloud even if more expensive) because I want to be on the safe side. What do you think, is this irrational and just a bubble thing?
It's a temporary crunch while TSMC expands capacity. They were reluctant to invest in capacity during the crypto crunch because they (rightly) understood that to be a temporary spike in demand. This time, they (also rightly) recognize AI as a longer-term shift in demand, and they're busy retooling to meet it.
The real question is whether NVIDIA will maintain its lock on the market, or if vendor-agnostic Torch will help commoditize the segment.
Cuda runs on a lower layer than torch. There is not much competition to cuda if one wants performance, but if one just wants to run it there is cpu (openblas etc and Intel's MKL) and onnx.
It's pretty astonishing to me AMD has neither a proper math library (like MKL) nor gpu compute library (like cuda).
When looking at cloud GPU availability and current trends (no one except some enthusiasts and bigtech is finetuning and serving on a large scale yet and results keep getting better and better), I fear we will run into a situation where GPUs will be extremely expensive and hard to come by until supply catches up?
I ordered a high end PC with 4090 for the first time in years (normally would always prefer cloud even if more expensive) because I want to be on the safe side. What do you think, is this irrational and just a bubble thing?