We are at PCIe5 in the Dell XE9680. We add in 8x400G cards and they talk directly to the Network/ 8xGPUs (via rocev2).
800G ethernet is here at the switches (Dell Z9864F-ON is beautiful... 128 ports of 400G), but not yet at the server/NIC level, that comes with PCIe6. We are also limited to 16 chassis/128 GPUs in a single cluster right now.
NVMe is getting faster all the time, but is pretty standard now. We put 122TB into each server, so that enables local caching of data, if needed.
All of this is designed for the highest speed available today that we can get on the various bus where data is transferred.
I wonder if any of this trickles down into cloud providers reducing costs again. After all if we have zounds of fast storage, surely slower storage becomes cheaper?
We do not directly compete with them as we are more of a niche based solution for businesses that want their own private cloud and do not want to undertake the many millions in capopex to build and deploy their own super computer clusters. As such, our offerings should not have an impact on their pricing. But who knows… maybe long term we will. Hard to say.
800G ethernet is here at the switches (Dell Z9864F-ON is beautiful... 128 ports of 400G), but not yet at the server/NIC level, that comes with PCIe6. We are also limited to 16 chassis/128 GPUs in a single cluster right now.
NVMe is getting faster all the time, but is pretty standard now. We put 122TB into each server, so that enables local caching of data, if needed.
All of this is designed for the highest speed available today that we can get on the various bus where data is transferred.