> a Python monolith that has one route that is CPU-bound and would otherwise be an ideal use case for a language with real multithreading
Sounds nasty. The classic python answer is to try rewriting the innermost loops in cython/c/cppyy etc or try using numba/pypy.
> best language for the job
The problem with this "best ... for the job" phrasing is that there's an objectively, indisputably right answer, where I've experienced atrocious choices being inflicted upon a team under that guise, resulting in a component that nobody wants to maintain because it's foreign to everyone.
> Sounds nasty. The classic python answer is to try rewriting the innermost loops in cython/c/cppyy etc or try using numba/pypy
Yep, we’re doing this. The problem is that we’re still calling back and forth between C and Python very frequently and rewriting that next layer in C or similar is prohibitively unmaintainable.
> The problem with this "best ... for the job" phrasing is that there's an objectively, indisputably right answer, where I've experienced atrocious choices being inflicted upon a team under that guise, resulting in a component that nobody wants to maintain because it's foreign to everyone.
Fine, then let’s at least not use the wrong tools for the job, in this case, Python and language’s that impose similar tradeoffs.
Yes - the trick really is to move your accelerated code away from python's object model, because then you can drop a lot of overhead and also release the GIL. Tricky, though. Out of interest, did you see any improvement from pypy or numba?
> prohibitively unmaintainable
I don't think well written cython should be any more unmaintainable than Go IMHO.
We didn't try numba and couldn't get pypy working; we had dependencies that didn't work with Pypy. I'm curious about numba though.
> I don't think well written cython should be any more unmaintainable than Go IMHO.
I've written a lot of Go, it's always struck me as about as maintainable as Python (I would put it at a bit more maintainable than Python given its mature static typing story--mypy still has lots of problems). I haven't given Cython a fair shake, but it doesn't seem very well-invested in; this is generally what has kept me away from it.
Sounds nasty. The classic python answer is to try rewriting the innermost loops in cython/c/cppyy etc or try using numba/pypy.
> best language for the job
The problem with this "best ... for the job" phrasing is that there's an objectively, indisputably right answer, where I've experienced atrocious choices being inflicted upon a team under that guise, resulting in a component that nobody wants to maintain because it's foreign to everyone.