Hacker News new | past | comments | ask | show | jobs | submit login

Its getting rather complicated to select a framework for high performance computation. But it is also more important than ever before because the automatic speed upgrades of CPU's are long gone.

Ideally one would like a library that checks at least all of the following (in no particular order):

* open source with a good community around it

* written in popular, modern, easy to use programming language

* offers complete functionality: linear algebra (including ndarrays), special functions etc. Numerical Recipes is a reasonable checklist of what is needed

* has an intuitive, "math-like" API that expresses well mathematical expressions

* makes optimal use of all available hardware (multi-core, GPU, cluster) without the need for (too much) specialized programming or tuning

* can integrate easily with other programming languages / frameworks as part of bigger projects

Those days there are quite a few candidates, from numpy/python, to armadillo,eigen,blaze,xla/C++ to various rust and julia libraries. What we don't have is an easy way to select :-). On the plus side, if the project is still in the early phases / not too big, it is relatively easy to try out and migrate. But a detailed and updated comparison table (e.g. in Wikipedia) would indeed be quite handy.




> Those days there are quite a few candidates, from numpy/python, to armadillo,eigen,blaze,xla/C++ to various rust and julia libraries. What we don't have is an easy way to select :-)

They all use or can use the same core libraries to do the actual computationally intensive parts (BLAS, LAPACK) so they mostly share the code where optimisations would make the most impact. So, at the end of the day, it's mostly about the API you like because performance-wise it's likely to be a toss up.


in principle there is this low level convergence but differences quickly pile up at the user level. eg armadillo does not support ndarrays except in an indirect way via "field" objects. 2d and 3d fixed size arrays are in general better supported by all libraries as they have wider applications in viz and games but arbitrary size / dimension tensors are hit and miss (e.g. in eigen they are an "unsupported" extension)

it gets even more complicated if you expand into efficient random number generation, graph processing etc. then it becomes important that the library is extensible with some plugin mechanism etc. as, ideally, you don't want your project to be a potpourri of different libraries / API's / dependencies

all in all there are some amazing projects out there but sometimes more options means it is harder to decide without spending serious time. I think this is one advantage of the python scipy stack: at the expense of a (possibly small) performance penalty you have one choice only but it is fairly complete and consistent


> easy to use programming language

> C++




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: