I agree that 90% of people do not need that, but there is a scale between 1 machine and 1 million machines.
It is not just IOPS but latency. Some of my production servers are connected one-to-one by crossover cable, in clusters with as many extra nics as needed (say 3 machines: 2 extra nics per machine) just to shave that little extra overhead of going through a router as yes, it matters in some applications!
Try using fiber then. I was surprised and pleased to learn that my 10G fiber has something like a third or quarter of the latency of my 10G copper in my new house.
You mean use (X)SFP(+), not “use fiber”. The SFP+ copper twinax cables I use in my lab at home have lower latency than the majority of MMF/SMF transceivers on the market.
Yes, thank you for the clarification/nomenclature. My SFP+ on MMF (multi-mode fiber) has significantly lower latency than the copper 10GbaseT I use. I haven't measured the direct-connect copper (coax? twinax?) SFP+ latency, though.
I would be curious if the SFP for 1G using fiber also has lower latency than copper 1000baseT.
It is not just IOPS but latency. Some of my production servers are connected one-to-one by crossover cable, in clusters with as many extra nics as needed (say 3 machines: 2 extra nics per machine) just to shave that little extra overhead of going through a router as yes, it matters in some applications!