This seems quite ridiculous to me, I have seldom seen "modern compilers are always faster than you" but rather "they are good enough that it is not worth it". It provides a very over-confident "conclusion" based on a single dubious test.
The main advantage of compilers is that the optimizations scale across a large codebase through inlining for example.
Also, just moving from Sandy-Bridge to Haswell for example can have significant performance swing (in both direction). The maintenance cost of the assembly is again a scaling issue.
If you have a single function that takes a significant amount of time in your program, and performance is critical, of course you can try to go with lower level. But it is likely that it will be more profitable to start with 1) pre-optimized libraries (i.e. don't write your own "sort") ; 2) follow the optimization guidelines of the CPU vendors regarding memory layout, etc. ; and 3) start with vector C-level intrinsic if possible if you can benefit from vectorization.
On the contrary, I've often seen the "you can't beat the compiler" statement. This[1] recent reddit thread has it in the top comment, which is what prompted me to test it out.
And while all those other points are fine points (and I mention all that in the conclusion), it doesn't change the fact that beating the compiler isn't always the rocket science it's made out to be.
Fair. As a compiler engineer (full-time on clang/LLVM), I wouldn't make such ridiculous claim (even though the compiler is capable of nasty tricks that "normal" humans wouldn't be able to pull).
Someone that pretends that it is not possible to beat the compiler should start by taking something like a GEMM routine (for example from there https://github.com/flame/blis/tree/master/kernels/x86_64 ) and reimplement it to show how a mainstream optimizing compiler can do better using C/C++ or Fortran.
You should always be comparing best case for this kind of thing. Slower cases are most likely "your thread got switched out by the OS to let something else run", and that's not really a fair test.
If you want to guard against context switches by the OS, don't use stuff that "most likely" works. Measure with perf and let it count the context switches.
It's totally ridiculous - I spent several years coding almost exclusively in x86 assembly and the prospect that a compiler could best a human at optimizing a specific function is ignorant. The only exception I can think of so-called 'superoptimization' where the optimal sequence of instructions is determined by exhaustively testing every combination. And that's not an effective strategy for anything beyond a handful of instructions.
Technically you could imagine some intelligent-agent based search algorithm that finds the near-optimal sequence of instructions for a given statement of some intermediate language, employing some heuristics derived from thousands of years of deep learning to get the search time down to something reasonable (say like hours or days instead of eons) - with the pressure on compilers today leaning towards everything being JIT'ed on the fly I don't think we're like to see it ever. It's just the old "sufficiently smart compiler" myth.
Depends entirely on the domain. Sorry, but i have never seen a programmer come up with the kinds of parallelizing transforms, cache blocking and iteration reordering, etc, that most polyhedral optimizers do.
Now, if you aren't doing these kinds of things, and are just trying to optimize the hot loop of some simple program, yeah, you can definitely win given enough time, because you'll just sit there with IACA or whatever, and superoptimize it by hand.
But you also are often starting with the output of a good compiler. If you had to start with nothing, i doubt you would do as well.
You speak as though all low-level optimization is alignment, unrolling, pairing, etc.. These are basically micro optimizations that yield negligible gains unless applied over a large code base. That Pluto output is just a wall of code because it has been unrolled several times, it's not any sort of impressive optimization achievement.
A human would probably convert this to fixed point, convert the entire inner-most loop into a couple of address operations and a single fused multiply add, process the array 64 bytes per iteration without unrolling anything. At that point it's probably 10-20x more efficient than that slab of polyhedral bullshit and finally he'd come back to carefully pad here and there to avoid misalignment penalties.
As for optimizing for 4 cores - you take your shiny hand-polished assembly routine and spin it up on 4 threads, most likely in high level code since you're talking to the OS to get threads, synchronize them, etc. It's not wise to chase parallelism at a low level because that goes counter to minimizing the overhead costs in setting it up.
> But you also are often starting with the output of a good compiler.
No, nobody does this. I mean maybe if you're just learning assembly. Starting with the compiler-generated garbage does not help you other than maybe by giving you a benchmark to beat.
> The main advantage of compilers is that the optimizations scale across a large codebase through inlining for example.
This. You will die of natural causes well before beating the compiler en mass, on a number of codebases I've worked with. And even when dealing with the tight inner loop, I often optimize with a mind to, not so much beat the optimizer, but aid it.
There are plenty of obscenely low level optimizations - optimizations I'd argue are operating at an even lower level than assembly - that one can apply without actually resorting to assembly. Avoiding cache line aliasing, for example:
I agree with the gist of your post, but the pre-optimised libraries are often optimised for the general case. It can be quite worthwhile to dive deeper in some cases. Replacing e.g. the library malloc with your own pool based allocator or a general hash table with a hand-rolled one where you exploit certain knowledge about the data can be huge wins.
You're right, but what you're describing seems to be on the order of "algorithmic changes" or "data structure changes" to be more suited to your use-case.
It seems to me that it is a higher level than what I was addressing: "I'll write my code in assembly to get it to run faster". Unless you're writing your pool based allocator in assembly ;)
It's less that "you can't beat the compiler", but rather that "trying to beat the compiler can backfire if you don't take a great deal of care". Some optimization advice doesn't apply to current compilers or systems, and can produce actively worse results. The worst-case scenario isn't "you did no better than the compiler", it's "you did much worse than the compiler".
The main advantage of compilers is that the optimizations scale across a large codebase through inlining for example.
Also, just moving from Sandy-Bridge to Haswell for example can have significant performance swing (in both direction). The maintenance cost of the assembly is again a scaling issue.
If you have a single function that takes a significant amount of time in your program, and performance is critical, of course you can try to go with lower level. But it is likely that it will be more profitable to start with 1) pre-optimized libraries (i.e. don't write your own "sort") ; 2) follow the optimization guidelines of the CPU vendors regarding memory layout, etc. ; and 3) start with vector C-level intrinsic if possible if you can benefit from vectorization.