Compiling a subset of Python syntax to x86-64 assembly for fun and zero profit

[benhoyt]

http://benhoyt.com/writings/pyast64/

[MichailP]

There was a whole course (notes for which are here [1]) dedicated to this topic, by prof. Jeremy Siek.

[1] ecee.colorado.edu/ecen4553/fall09/notes.pdf

[benhoyt → MichailP]

Wow, that looks really good -- thanks for sharing the link. Yeah, it's definitely similar to what I'm doing but in a more formal and non-toyish way.

[denfromufa]

also worth mentioning here llvmlite & numba, peachpy:

http://llvmlite.pydata.org/en/latest

http://numba.pydata.org

https://github.com/Maratyszcza/PeachPy

[john_moscow]

Looks cool. But why asm? Generating C or C++ code and letting the GCC optimizer do the heavy lifting for you could probably get you a massive perf boost.

[viraptor → john_moscow]

Cython already does this. Including using type hints.

[ianai → john_moscow]

Because the people who talk down python for not being c/c++ "would win".

[john_moscow → ianai]

Why? Both C/C++ and Python are just tools and they have different strong/weak sides. I can't see much harm in combining the strong sides of 2 worlds.

That said, isn't PyPy (https://pypy.org/) already doing the same thing?

[ianai → john_moscow]

I wasnt being serious - in line with the headline "for fun and no profit".

Optimizing python with calls out to optimized c/c++ code has always been a thing. I think a large part of the interest here is novelty.

[soravux]

Interesting topic. We did something similar in the past using the bytecode instead of the AST tree to accelerate genetic algorithm evolution:

http://multigrad.blogspot.com/2014/07/low-level-frenzy.html http://multigrad.blogspot.com/2014/06/fun-with-python-byteco...

I'm curious, what was the original use case for pyast64? Feels like we could apply it to genetic programming fairly straightforwardly.

[benhoyt → soravux]

Sorry, I didn't have a use case -- I did it purely for fun. However, I got the idea when I was trying to optimize some Python bytecode.

[senorsmile]

This was a very fun read. Makes me want to get back into assembly, figure out 64 bit assembly and make this more than a toy... almost.

[mtanski]

> pyast64 with peephole 0.24 29.7

> C version (gcc -O0) 0.22 31.8

Modern x86_64 processors are really good at ruing "bad" asm pretty efficiently.

[alephnil → mtanski]

Absolutely. The amount of tricks used to achieve that is amazing, including things register renaming, branch prediction, virtual registers, instruction reordering etc.

That is not really what demonstrated here though, as gcc was run with optimizations turned off, since otherwise the program would have been optimized to printing a constant.

[ponytech]

Using the same technique, would it be possible to generate WebAssembly ?

[ra7]

Informative article as always from you. Thanks!