Getting the theoretical peak performance requires either extensive manual tuning on each architecture or exhaustive search. To read on the former see http://www.nytimes.com/2005/11/28/technology/28super.html?pa..., and the latter is done for example in the ATLAS BLAS.
It is important to note that the SO discussion is focused on achieving peak flops when the data is already in the XMM registers. The real bottleneck is pushing the data quickly through the levels of the memory hierarchy.
I was surprised by how little low-level knowledge it took to get within a stone's throw of the theoretical peak.
The general rule seems to be:
- keep data in packed registers (use YMM instead of XMM if you have them)
- interleave addition and multiplication
- unroll the loop sufficiently to avoid stalls
I wonder if how close to ATLAS you could get if you wrote a matrix multiply following these principles and additionally structured the row/column iterations into cache-sized tiles.
CPUs are very capable today, unfortunately, to get 'every last drop of performance' is very difficult.
I'd say that CPUs are more complex than compilers can optimize the code for them.
Of course, manually optimizing for something like this is "easy" (if you have a somewhat deep knowledge of Intel's manuals)
Now for "everyday computing" this gets tougher, even though compilers do a good job (good, not great) and it's usually good enough.
So you end up going to the deeper details in time-sensitive things: games, video processing, etc
There are some tools (from Intel and AMD, even though I tested only the AMD one some time ago) that tells you 'everything' that you need to know about how good your code is. For example, IIRC, if you load a register then immediately store to it (or store then load, I don't remember), there's a stall, so you can do something else in between
I'm a bit surprised by the most voted answer by a user who has 80K rep on SO.
By this part:
"If you decide to compile and run this, pay attention to your CPU temperatures!!!
Make sure you don't overheat it. And make sure CPU-throttling doesn't affect your results!"
Fair enough for the CPU throttling.
But did the nineties just call? Computers dying due to CPU overheating was all scary and spooky back when we were running old AMDs and old Pentium CPUs but I haven't own any CPU that could die due to overheating since a very, very long time. They do automagically throttle to: a) make sure they stay within their TDP specs b) make sure they don't melt.
I mean: honestly, scientists should start to worry about overheating their CPU that are going to melt?
I think the only thing in favor of the argument that it might overheat the CPU is that many more functional units on the CPU will be active at once and with no time in between. Normally they would be idle due to cache misses and memory access.
I don't think with modern CPUs that this is a problem, but you would have to test to find out!
Outside of leakage current CMOS/MOSFET circuitry draws current during state transitions. So, yes, more bits flipping per unit of time will cause additional heating, sometimes dramatically so. This is the reason the last generation of iPads get so hot: Four tomes the pixels means that you have the potential to have four times more bits flipping in the display processing pipeline, hence more heat. Try running footage where every pixel is changing on every frame and see what happens.
>I mean: honestly, scientists should start to worry about overheating their CPU that are going to melt?
My officemate is working with some neural networks researchers, trying to get near-peak throughput running convolutional neural networks on NVIDIA's new K20 GPUs. Several GPUs have already died and they're looking for better cooling for the rest. So, I don't know if peak utilization of multi-core CPUs can still be hazardous, but it's definitely true for graphics cards.
It is important to note that the SO discussion is focused on achieving peak flops when the data is already in the XMM registers. The real bottleneck is pushing the data quickly through the levels of the memory hierarchy.