Die space. There's a maximum reticle size. You can't make chips bigger than that without using multi-chip techniques like wafer scale integration, chiplets and so on.
If you do a full reticle chip and use most of it for memory you can get about 1GB of SRAM. But that would also be extremely expensive.
It's not quite as bad as you might think (though it is bad).
TSMC N5 is 0.021um per SRAM cell in high-density configuration (still going to be 5-10x faster switching than DRAM). That's about 47.62 bits/um. With 1,000,000 um2 per mm2. EUV reticle limit is 858mm2.
That amounts to 40,857,142,857 bits or about 4.756GB. That's almost exactly 250 dies per wafer. TSMC N5 wafers cost $17,000 or roughly $68 per chip (you can get very close to 100% yields on this kind of very basic chip). If we use part of the chip for internal controllers and interconnects, it would be about $70 per 2-4GB.
The highest-clocked DDR5 RAM I could find with minimal searching was $360 for 32GB.
The SRAM version would cost $560 at the cheapest for just the 32GB RAM not counting the RAM controller chips and all the packaging costs. With profits and everything included, your final price tag would likely be around $750 if you could produce in bulk and $1000 if you could not.
On the plus side, worrying about stuff like CAS latency would basically be a thing of the past with the bottleneck once again becoming the speed over the wire. Power consumption would also be lower without the need to constantly refresh. I doubt that anyone wants to pay that much money for disproportionally marginal increase in performance, but looking at all the people forking over thousands to scalpers for GPUs, maybe I'm wrong.
You might want to check your maths on that. It's more like 30 maximum reticle size dies per wafer IIRC. Just the area based limit is pi * 150^2 / 858 = 82, but you obviously lose a load because dies are rectangular and wafers are circular.
If you do a full reticle chip and use most of it for memory you can get about 1GB of SRAM. But that would also be extremely expensive.