prometeo is an experimental modeling tool for embedded high-performance computing. prometeo provides a domain specific language (DSL) based on a subset of the Python language that allows one to conveniently write scientific computing programs in a high-level language (Python itself) that can be transpiled to high-performance self-contained C code easily deployable on embedded devices.
The package is still rather experimental, but I hope this concept could help making the development of software for high-performance computing (especially for embedded applications) a little easier.
What do you think of it? Looking forward to receiving comments/suggestions/criticism :)
The benchmark in prometeo is a discrete time Riccati recursion (as opposite to a continuous time Riccati equation) algorithm. And it is the exact same algorithm implemeneted in all languages, making the comparison more fair as the only variable is the implementation itself.
Kneejerk reaction as an enthusiastic Cython developer: "bah, another crappy (subset of Python)-to-C compiler."
After reading: this is really cool. If I understand this, I think you should be able to beat Cython without breaking a sweat. I'm quite excited to use this.
Looks like RPython is a bigger language that doesn’t target an embedded use case without a Python runtime. Though I may be mistaken - I am not super familiar with RPython.
To get ahead of the obvious question I had and I’m sure others will, this is from the README:
> Cython is a programming language whose goal is to facilitate writing C extensions for the Python language. In particular, it can translate (optionally) statically typed Python-like code into C code that relies on CPython. Similarly to the considerations made for Nuitka, this makes it a powerful tool whenever it is possible to rely on libpython (and when its overhead is negligible, i.e., when dealing with sufficiently large scale computations), but not in the context of interest here.
I.e., it’s a python-like DSL that does not depend on the Python runtime.
Right, that's indeed the main reason I could not simply use Cython or Nuitka (or Julia?). The Python runtime library will do all kinds of non real-time/embedded friendly operations such as garbage collections, memory allocation/de-allocation and so on, in the background.
This is awesome! The direction of using a subset of python, while leveraging the user base and static typing to accomplish some other everyday task in a different language is very legit IMO.
I'm not spending much time on py2many last few months (started a new job). Let me know if any of it sounds useful - especially the ability to transpile to 7-8 languages including Julia, C++ and Rust.
No intermediate AST. To understand the various stages of transpilation and separation of language specific and independent rewriters, this file is a good starting point:
Thanks for the question! My background is in numerical optimization for optimal control. Projects like this https://github.com/acados/acados motivated the development of prometeo. It's mostly about solving optimization problems as fast as possible to make optimal decisions in real-time.
Have you considered using Nim? It's a great language that has some similarities to python and compiles to C code. It is a very convenient and powerful language to use, coming from Python.
Well, you give yourself a whole lot of extra work by attempting to create a Python to C transpiler, basically creating an even less mature and less popular language ecosystem than Nim has, when you could use Nim and get most of the benefits (statically typed, compiled to C) out of the box and enjoy their growing ecosystem. It seems that Prometeo is currently more limited than Nim.
Maybe I'm not understanding your use case well enough and maybe your approach is actually a locally optimal solution, and I honestly greatly respect your effort and want to see your project succeed, cuz I love Python too. I even wanted to make a faster python myself at some point. Just, the more I learn about Nim, the more I appreciate its design decisions and think to myself: this is the faster version of python I'm looking for. Now, we just need the large package ecosystem that python has, but I'm willing to both wait and participate in making it come about.
This presentation about writing keyboard firmware in Nim may be helpful, if you're willing to give Nim some more consideration:
I noticed your disclaimer at the bottom of the linked page [0], and wanted to get an idea of how far you were looking to take this. Will it go beyond maths into normal functions (string handling, etc) ? Do you eventually plan on supporting most of python - for instance, do you think I could write a web server using your tool in the future?
[0] - "Disclaimer: prometeo is still at a very preliminary stage and only few linear algebra operations and Python constructs are supported for the time being."
Unfortunately, I think that writing a transpiler for general Python programs might be rather difficult without resorting to approaches used, e.g., in Cython/Nuitka. Among other things, computing the worst-case heap usage could be quite cumbersome/computationally heavy for a general program without "constraints". I'd be happy to hear what others think about the topic though.
Soo... it takes Python syntax and produces a C program, with no links back to Python - is that right? It uses a strict subset of Python, so that Prometeo programs are valid Python, but not necessarily the opposite. Is that fair?
Do you envisage this being a conduit for tight loop optimisation in Python? Or is it rather "you'd like a C program but can't write C good"?
And if the former, how do you compare to Nuitka and Cython? I read your README but couldn't quite make sense of it :)
> Soo... it takes Python syntax and produces a C program, with no links back to Python - is that right? It uses a strict subset of Python, so that Prometeo programs are valid Python, but not necessarily the opposite. Is that fair?
yep
> Do you envisage this being a conduit for tight loop optimisation in Python? Or is it rather "you'd like a C program but can't write C good"?
There are already plenty of options for calling high-performance libraries from Python. Now 1) interpreting Python programs that use, e.g., NumPy, can be slow. 2) Compiling these programs using, e.g., Cython or Nuitka, can speed up the code across calls to high-performance libraries, but the resulting code will still rely on the Python runtime library, which can be slow/unreliable in an embedded context.
Coming to the second part of the question, writing C code directly is definitely an option, but, after doing a bit of that, I realized how tedious/error prone it is to develop/maintain/extend relatively complex code bases for embedded scientific computing (e.g. this one https://github.com/acados/acados). Or, to put it as Bjarne Stroustroup once said "fiddling with machine addresses and memory is rather unpleasant and not
very productive". The good news seemed to be that many of the code structures necessary to write that type of code are rather repetitive and can hopefully be generated automatically to some extent.
> And if the former, how do you compare to Nuitka and Cython? I read your README but couldn't quite make sense of it :)
This table (from the README) shows some computation times for Nuitka, prometeo, Python and PyPy.
Right. Gotcha. So Prometeo isn't another "make Python fast again" project, but rather an orthogonal effort to write fast (C) programs, but in a high-level Python-like language. Thanks.
C code is performant and that is a fact.
Python code is not.
When building mission critical systems why don't programmers just use C itself instead of coding in another programming language and having it transpiled for them? Why introduce such tools all the time?
I am against this because the tools programmers use are becoming too bloated compared to 10-20 years ago.
Want to build an Android App? Use Java/Kotlin.
Want to build an iOS App? Use Swift.
Want to build a Web App? Use a Single JS Framework (Why millions of frameworks?)
Want to build a Windows Desktop App? Use C#.NET Either with WinForms or WPF.
I really see tools and technologies coming up all the time to solve a problem that most of the time doesn't exist.
You've never been near a lab environment clearly. Python is a dominant language in university labs and runs of lot more real-time systems than you think. Grad students rarely have industry experience and don't necessarily have the know-how to write C code effectively, so it's a question of resources and ecosystem. Numpy, matplotlib, pandas, scikit, TensorFlow, etc. are all huge draws for the scientific and ML communities.
To save some reading, this uses a combination of an FPGA and kernel (which I think is in Rust) to generate a realtime buffer that is programmed using a mix of compiled-realtime python and plain python via a RPC system.
We can talk instead about how the requirements to run slightly less simple near-hard real-time controllers are really heavy in terms of money, effort and .. weight. This tool may actually help to streamline the software part, potentially being a substitution in certain cases to i.e. Matlab Coder or similar tools.
I think many people who have at least once first prototyped a numerical algorithm in a high-level language (say Python, Julia, MATLAB?) and then implemented it in C, can relate to the experience of transitioning from error messages of the type: "dimension mismatch for XYZ" to "segmentation fault". That's in my opinion a strong motivation to build tools that can automate certain parts of the development process.
Writing C code directly is as a good option, as long as your code is not too complex to develop, maintain and extend.
And, again, Python here is intended to be the host language for an embedded domain specific language that gets compiled into C. It does not need to be efficient it needs to be expressive and easy to analyse and transpile.
Note that the whole point of Julia is that it saves you the rewrite. There is Julia code running on top supercomputers that gives speed competitive to C/C++/Fortran. You will have to put in some work to get Julia code to be that fast, but it is usually dramatically easier than a rewrite in a different language.
yes, but "speed" in 'top supercomputers' is not "speed" in 'embedded systems'.
I do think Julia potentially can crack this space, but the runtime at least historically has not been tailored for it.
It does seem like Julia has become more modular lately especially being able to disconnect the JIT (or LLVM ORC). H Hopefully you'll either being able to either defatten or completely remove the runtime dependencies (ala Rust in no-std mode). Each of these is important for different use cases.
The problem that this language solves is that it automatically sorts out the memory usage for you. That isn't a problem for me; I've been programming in C for decades. But it is a problem for most python programmers who don't have a lick of C experience, but want to get C performance. It drastically lowers the barrier of entry.
For what it counts, I have developed code for this kind of applications exclusively in C for ~5 years (let's say 20% of my working time). I still think that debugging a segfault that you could have avoided is not very productive and that motivated me to look into possible alternatives.
FWIW, I almost always use valgrind before a debugger, when tracking down segfaults. It doesn't catch everything, but 90% of the time, it gets me to the right region of code in a single run.
Good good, just wanted to advocate for my favorite tool there. But, in my experience, segfaults are usually the easiest bugs to resolve. Unlike a sign error in my math, they're impossible to miss!
That said, tooling to get rid of them entirely is not to be sneezed at :)
if we are arguing that implementing a numerical algorithm in C is as easy as implementing it in Python - I would disagree. But maybe I am just wrong :)
It doesn't have to be production. Maybe it's for a research project where you just need the extra performance.
Everything has a cost. This may not be ideal but learning to do C properly as an experienced Python dev will have a time cost as well. This may just be the best way to get from A to B.
I remember when I did a one-off project with a PIC microcontroller. I only had an assembler and I spent 2 days getting nowhere.
Then i found a C compiler and I had the whole thing running in 2 hours. The compiler turned out to much more efficient in speed as well as code size than my hand-written assembler.
> When building mission critical systems why don't programmers just use C itself instead of coding in another programming language and having it transpiled for them?
I can't speak much about the code itself or the aims of the projects. Personally I would recommend more informative commit messages.
I do this myself, especially working on personal stuff, but writing commit messages that succinctly explain what each commit does is a good practice and gives a serious impression.
I often find myself hacking away and periodically going back to flesh out messages using rebase.
Thanks for the suggestion. Until now it's been a lot of discussion with friends and colleagues and much less actual collaboration on code writing - I might have drifted into bad practices.
This isn't that meaningful since the "Julia" version is just calls to OpenBlas/LAPACK which are known to have relatively high overhead for small matrices. I'd be much more interested in seeing a comparison vs LoopVectorization/StaticArrays which are Julia libraries specifically optimized for small matrices.
Right, for sure I would not need to re-invent the machinery to translate a class into a glorified C struct. The whole thing started with C in mind for portability arguments, but it might be a good idea to keep an eye on C++ as an option.
Have you thought of targeting WebAssembly? If you're going from Python/Prometo -> C you could always make the extra sep of Python/Prometo -> C -> WASM but I wonder if there would be an advantage of skipping the intermediate C.
Python to ASM would actually be really cool and would guarantee performance gains for small matrices, but it would require quite some implementation effort. Not sure about WASM.
Why WASM? It would be a pessimization compared to just transpiling to C if performance is the goal. WASM also is restricted to 128-bit vector instructions.
That would be running python in wasm so python (script) -> pyiodide (cpython/wasm) which is pretty heavyweight this would be prometeo -> wasm which I'd imagine would be fairly lightweight. If someone is willing to write in AssemblyScript I figured that something like Prometeo might be a welcome alternative. As I said if it transpires to C there's nothing stopping someone from using emscripted to then go to wasm.
One major problem is that the error messages need a lot of work. Why aren't class variables and static methods not accepted? I can't know that if your code just throws an exception while iterating some dictionary.
I agree that error handling is one of the main things to be improved. The problem is that in some cases the AST walker ends up in unhandled states and prometeo throws a generic exception with a line number only. Are you looking at something in particular? With basically 0 users at the moment, this kind of feedback is quite useful.
It seems a convenient/high_level way to use highly optimized C libraries with minimal overhead both in terms of execution time (i.e. vs standard interpreted Python) both in term of runtime size/complexity (see Julia).
That's correct. I'd say one of the fundamental differences between the two lies in the fact that the code generated by prometeo does not depend on a runtime library (which is somewhat fundamental for embedded applications, e.g., embedded optimization). From prometeo's README:
Finally, although it does not use Python as source language, we should mention that Julia too is just-in-time (and partially ahead-of-time) compiled into LLVM code. The emitted LLVM code relies however on the Julia runtime library such that considerations similar to the one made for Cython and Nuitka apply.
Nice job! Is this aimed at single core/thread computations or the prometeo layer is also a way to write in a more "user friendly way" basic parallel code?
Do you have access to builtins and intrinsics? Are there any plans?
The single threaded thing is not an issue because you can still call the same function on each CPU and use the CPU ID to target parts of the computation, like a compute kernel function.
Intrinsics (or directly assembly) are used in BLASFEO (https://github.com/giaf/blasfeo) the linear package used by prometeo. It would be cool to generate assembly directly for a few things, but that would require quite a bit of work!
Indeed, and that would make the code machine-dependent too. On top of that, BLASFEO is already used for the most computationally intensive operations (e.g. matrix-matrix operations and factorizations, involving O(n^3) flops on O(n^2) data) providing high performance and low overhead, and the other operations on vector and matrices are anyway usually memory bounded and would see little to no benefit from directly using the assembly language.
On top of linear algebra, is it also possible to interface it directly to standalone, well optimized existing QP solvers, like HPIPM (https://github.com/giaf/hpipm)? In case one is interested in implementing MPC controllers.
You are not the only one request this :p
So, if I understand correctly, at the end what you would like is a tool to write your MPC problem more conveniently in a high-level language and still be able to deploy your code using HPIPM on an embedded device?
[BTW, I happen to be the main HPIPM developer, so very glad about your comment!]
Stand-alone is a very useful concept. I don’t like deploying Python stacks much. Wouldn’t that additionally mean you could target CL, CUDA or Sycl variants of C?
Cython, pypy, micropython, nuitka, shedskin, ironpython, graalpython, jython, mypyc, pyjs, skuptjs, brython, activepython, stackless, transcrypt, cinder and many more I don't remember.
They're all practically useless or delegated to specific tasks. At this point you'd need to present incredible evidence that an alternative compiler can be useful. Personally I find it comical how many developers are still eluded by a promise of performant python. I hope you achieve your goals, good luck.
The point of prometeo is not to obtain a "performant Python". Python is used merely as a host language for an embedded domain specific language. You could do the same thing with any other language with a mature library for AST analysis :)
Most of the ones you list require dynamic linking and so is hard to make use of in specialized environments.
His project seems to be generating generic C code which is much easier to port to any weird platforms. In fact, it might be perfect for my use-case where dynamic linking is just extra attack surface.
I understand that the project is still in the early stages, but I will be paying close attention to it. If at some point it will be possible to write "regular" Python in it (minus most of the standard library and imports), then it could be a candidate for an edge computing platform.
Cython is also pretty successful obviously, though I don't think it quite fits in OPs list given that it's more about writing extensions than replacing your entire Python code/stack. But I do agree with OPs sentiment even as someone who writes a lot of Python.
I think most HPC people would disagree with this statement. State-of-the-art HPC code is still written in ASM (see e.g., https://github.com/xianyi/OpenBLAS) [that's what Intel is doing too]
ASM makes sense when the time spent in a specific routine exceeds the time it takes to write the ASM, which makes a lot of sense for Blas, less so for other HPC yet speculative or less fundamental projects. Cvodes for instance doesn’t need to be written in ASM, and I think Julia makes a strong case that it could have been written in Julia.
I don't think they would. I think they realize that state-of-the-art HPC code is a small fraction of all the code written. I doubt that these people write ASM instead of Python or JS or C or whatever when doing simple scripts.
That ASM code is however not necessarily constructed manually. You'd think for high performance code with limited scope, a superoptimizer would be used.
Not sure what a "superoptimizer" would look like in this context. For a reference, I know for sure that this https://github.com/giaf/blasfeo (which beats Intel MKL) was coded entirely by hand.
There is more and more effort in the automatic development of high-performance linear algebra kernels.
But based on my experience, it would certainly be a big challenge to have a tool able to exploit the subtle differences in the assembly languages of different architectures, if the aim is to match or even exceed an expert-crafted assembly kernel.
Anyway, that's surely a very promising active research direction.
Good point. And you don't have to go that low. Maybe go use Object Pascal, Nim, or Vlang. I know... the libraries. But a lot of them are bindings of C libraries. So, you can create bindings in other languages too or use Python from those languages. There are various options.
There's ongoing work to reduce runtime dependencies of Julia (for example in 1.8, you can strip out the compiler and metadata), but then it's only approaching Go/Swift and other static languages with runtimes.
Generating standalone runtime free LLVM is another path, that is actually already pretty mature as it's what is being done for the GPU stack.
Well IMO it can definitely be rewritten in Julia, and to an easier degree than python since Julia allows hooking into the compiler pipeline at many areas of the stack. It's lispy an built from the ground up for codegen, with libraries like (https://github.com/JuliaSymbolics/Metatheory.jl) that provide high level pattern matching with e-graphs. The question is whether it's worth your time to learn Julia to do so.
One cool use case is in https://github.com/JuliaLinearAlgebra/Octavian.jl which relies on loopvectorization.jl to do transforms on Julia AST beyond what LLVM does. Because of that, Octavian.jl. a pure julia linalg library, beats openblas on many benchmarks
Octavian isn't just better than OpenBlas. It beats MKL by about a factor of 2 up to 100x100, and is roughly tied with MKL up to around 3000x3000 (OpenBlas is 2-3x slower up to around 500x500)
Thanks for mentioning Octavian, I didn't know about this interesting project.
Are you referring to single- or multi-threaded applications?
In the context of embedded optimal control applications (i.e. the original framework motivating the Prometeo development), applications are typically single-threaded, and in this case for matrices of size 100x100 MKL is _very_ close to peak performance already, there is no way something can be 2x faster without breaking the laws of physics.
[Trust that I know what I'm saying here, as the main BLASFEO developer, I check MKL performance often enough ;) ]
Just for reference, MKL has special flags MKL_DIRECT_CALL and MKL_DIRECT_CALL_SEQ which enable extra optimizations improving performance for small matrices (e.g. turn off most input arguments checks), these should definitely be used in a fair comparison.
On top of that, linear algebra is much more than matrix-matrix multiplication, and e.g. in embedded optimal control the performance of factorization routines plays a key role.
Octavian is absolutely early in it's development (currently I think it only supports matmul including all the transposed versions). https://raw.githubusercontent.com/JuliaLinearAlgebra/Octavia... is the benchmark. It uses automatic threading from both MKL and Octavian (although for these sizes, it will only use a few threads). With only one thread, MKL is much closer and is only behind by about 20% at n=25 and roughly equal by n=60. I haven't done timings with MKL_DIRECT_CALL or MKL_DIRECT_CALL_SEQ, but I think that's unfair since Octavian has the same overhead of figuring out how many threads to use.
Looking forward to see Octavian development then, it looks exciting! Dealing with triangular matrices and data dependencies in other linear algebra routines such as triangular solves and factorization will surely be an interesting benchmark for the approach, since such difficulties do not arise in matrix-matrix multiplication. Anyway, that's surely a good starting point for Octavian.
Just one clarification: MKL_DIRECT_CALL or MKL_DIRECT_CALL_SEQ is not about figuring out how many threads to use, it's about turning off checks on input arguments sizes, e.g. if m>lda, or negative lda or m or stuff like that. All these pedantic checks (which comply with the reference BLAS implementation in Netlib) are often times not done anyway in experimental linear algebra packages that do not aim at providing a compliant implementation of the standard Fortran BLAS.
I though about using Nim as a host language for the DSL for a while, but then decided to rely on Python simply because it is more mature (and I had already partially figured out how to manipulate Python ASTs to generate C code).
"A program that translates between high-level languages is usually called a source-to-source compiler or transpiler" from https://en.wikipedia.org/wiki/Compiler.
Anyway, the main point is to give it a name that is as informative as possible. If I read "transpiler" I immediately make the connection with the fact that it translates a high-level language into another high-level language (which is what prometeo does)- but maybe I am biased.
Not sure how easy it would be to make the code generated by Pythran standalone, i.e., no dependency on the Python runtime library. Any Pythran expert? :)
BTW very cool project nevertheless, just wanted to see the differences parallels to pythran. There might even by room for collaboration on some features.
No, that's not the timescale of interest, I would say. However, if the big chunk of computations is delegated to HPC libraries I would say that NumPy could be rather competitive there (although still not easy to embed). If instead you need to run many times the same piece of code where a large fraction is pure Python, of course, it would not change the picture with respect to the "small" computations scenario.
It's kinda of sad how much effort is put on the creation of new Python compilers to make it slight faster while the problem of latency to compile that people hate at Julia is not tracked because of the lack of manpower to improve Julia's interpreter.
https://youtu.be/IlFVwabDh6Q?t=2530 (tldr: The Julia interpreter is currently about 500x slower than JIT code and there are a lot of low-hanging fruit work there that could easily give it a 10x speedup - this could make more viable to switch between compiler and interpreter depending on the work)
Personally, I think Julia is great - just don't know it well enough to write a package that takes Julia ASTs and generate C code from them :) There could totally be a Julia implementation of the main idea behind prometeo (Julia per se does not solve the problem that prometeo aims at solving).
The LLVM code will only call into the runtime for allocation or dynamic dispatch, both of which are avoidable. Lots of real Julia code will never touch it.
the problem with Julia in the use case of OP is really the fact that it is garbage collected (and perhaps also how its GC is tuned). You can work to eliminate allocations, but the memory determinism problem is more important in real time control and embedded systems. see for example, this video: https://www.youtube.com/watch?v=dmWQtI3DFFo
just write your code in c or c++ and be done with it. if you need math libs there are plenty out there for anything you can imagine. python will go the way java went many years ago.
prometeo is an experimental modeling tool for embedded high-performance computing. prometeo provides a domain specific language (DSL) based on a subset of the Python language that allows one to conveniently write scientific computing programs in a high-level language (Python itself) that can be transpiled to high-performance self-contained C code easily deployable on embedded devices.
The package is still rather experimental, but I hope this concept could help making the development of software for high-performance computing (especially for embedded applications) a little easier.
What do you think of it? Looking forward to receiving comments/suggestions/criticism :)