To be fair, the sparse matrix package is still very rough. I am highly skeptical of the 1sec in java vs 8 hours in scipy, though.
In general, numpy/scipy is quite faster than R: it does not have the pass by value semantics for once. I am also skeptical about writing a "new" R: the main value of R is in the R packages. Any new language would threw that away.
thanks, I will take a look at it at home (where I have enough memory to deal with that kind of matrix). As I said, the scipys.sparse is pretty rough and too low-level, but 3 sec vs 8 hours is way too long to be normal.
(I forked your gist so you know how to contact me)
In general, numpy/scipy is quite faster than R: it does not have the pass by value semantics for once. I am also skeptical about writing a "new" R: the main value of R is in the R packages. Any new language would threw that away.