On: The LLVM Compiler Infrastructure

[jclay]

For those who haven't explored LLVM, it's an incredibly powerful technology and rather approachable for getting started with.

A few of my favorite LLVM discoveries:

- If you want to build Clang (LLVM C/C++ Compiler), it's really simple to pull down and build with little external development tooling (only a compiler and CMake). Shown here [0].

- You can pull LLVM/Clang master and use that for a "bleeding-edge" toolchain with a high degree of stability. [0].

- Clang is used to compile Chromium and Firefox on all platforms. Both have seen great performance gains from Clang's link time optimization. [1] [2]

- Klee is built on LLVM infrastructure and is used for automatic generation of test cases. [3] See some of their papers for how powerful their results are.

- Clang can now produce ABI compatible binaries with MSVC and has support for the Visual Studio debugger. [4]

- Lots of interesting research projects leveraging LLVM [5]

[0] https://www.youtube.com/watch?v=uZI_Qla4pNA

[1] http://blog.llvm.org/2018/03/clang-is-now-used-to-build-chro...

[2] https://glandium.org/blog/?p=3888

[3] https://klee.github.io

[4] http://blog.llvm.org/2017/08/llvm-on-windows-now-supports-pd...

[5] https://scholar.google.com/scholar?as_ylo=2017&q=LLVM&hl=en&...

[eslaught → jclay]

Klee is also a great example of the cost of LLVM's never-backwards-compatible approach: according to their home page, the stable release of Klee runs on 3.4 and 3.8 is "experimental". (The LLVM releases since then have included: 3.9, 4.0, 5.0, 6.0, and 7.0.)

Don't get me wrong, LLVM is great, but I really wish that as the project matured they had made more of an effort to provide some level of compatibility. As it is, it's a massive pain for open source projects to follow along.

[drmeister → eslaught]

Counterpoint - I implemented a Common Lisp compiler that uses llvm as the back end (https://github.com/clasp-developers/clasp). I've kept up with the versions from 3.6 to 6.0 and I'm about to upgrade to 7.0.

I'm not sure why other projects find this difficult but I have noticed that if I get too far behind it does get a bit more difficult.

I exposed the C++ API to Common Lisp and write all of the compiler in Common Lisp. The Clang C++ compiler tells me what is broken when I upgrade to a new version of llvm and then I fix it. I talk about it here https://www.youtube.com/watch?reload=9&v=mbdXeRBbgDM&feature... at about 13:30

[jcranmer → eslaught]

If you compile a bitcode file from anywhere around LLVM 3.3 or later, you can generally read it in the most up-to-date version of LLVM (you may have to first strip debugging metadata though).

The LLVM-C API also maintains compatibility, but that comes at the steep cost of preventing you from using most of the power: LLVM-C is basically limited to reading/writing LLVM code, running passes largely at the -O1/-O2/-O3 level of granularity, and executing the JIT. Writing a pass with that API is very difficult, and pretty much impossible if you want to use any analyses.

[jclay → eslaught]

That's a good point. Most of the large LLVM projects I've used (such as Root's Cling) seem to lag behind LLVM's versions rather drastically.

[jclay → jclay]

My list is mostly C++ focused, but the LLVM tooling for clang-tidy/clang-query is also worth a mention. There are a big set of refactors that you can apply across your codebase if you wanted to migrate to new C++ features for example. You can also write your own if you wanted to refactor to use the API of an updated dependency for example. It's a lower-level version of Javascript's "codemods" for C/C++.

There's some work being done to build some tooling around interactively creating AST matchers and applying refactors across an entire codebase. [0]

[0] https://steveire.wordpress.com/2018/11/11/future-development...