On: Google ML Compiler Inlining Achieves 3-7% Reductio...

[quotemstr]

In general, every heuristic in systems programming --- kernels z compilers, databases, whatever --- is an opportunity to substitute an ML system.

[brrrrrm → quotemstr]

today, learned heuristics have a couple of pitfalls that make them hard to add to such systems

1. they are usually hard to run efficiently

2. they are usually hard to explain

The former is definitely changing with low precision formats like fp16 and useful coprocessors that can do matrix multiplications efficiently (M1, Intel). The latter hasn't been developed much and unless you're just training a model to memorize the entire space the heuristic operates in, it can be scary to trust it on unseen data.

[urthor → brrrrrm]

From my work, I've found it's really simple and straightforward to apply.

1. Choose a parameter for your compiler, xxx.

2. Have your ML model "choose compiler config parameter yyy." After the ML model "chooses" the config parameters, work backwards.

3. Determine why yyy is a better config parameter than xxx.

It might not be!

This system works, brilliantly. Cyborg intelligence, a combination of the human being and the ML model, is the future of society.

The key is the ML "suggests." ML must keep "suggesting."

Never have ML choose a parameter autonomously.

That's exactly how you get self driving cars running over children.

[moonchild → urthor]

> Choose a parameter for your compiler, xxx

Most interesting cases don't really look like this. The heuristic is applied to the user's code; it's not a one-time knob in the compiler. If it were, then you would likely be able to afford an exhaustive search to pick it & wouldn't need ml.

[urthor → moonchild]

Depends on the domain.

The analogy I'm making doesn't especially apply to compilers, which have human defined defaults in the first place.

What I've found is that it's important to not run the ML model then use its output as the default state. Have a human, heuristic choice as the default state.

[amelius → brrrrrm]

Also, they are not stable. I.e, if you have a fast program and you change a tiny detail, it is not guaranteed that the program remains fast. Also between versions of the compiler.