On: Backend of Meta Threads is built with Python 3.10

[ska]

Is your problem that the c part of GDAL isn't concurrent and you are calling it multiple times, or are you blocking on something you shouldn't?

Python calling a concurrent c thing is super common. For more complex things there is multiprocessing, but that essentially is simplification of IPC.

[pookha → ska]

Had a massive amount of images that I was trying to process for a specific purpose. I wound up spinning up multiple python processes within a java wrapper using concurrent linked queues. Got the job done but it's not the proudest moment of my life.

[seunosewa → pookha]

Why did you need the Java wrapper? You can orchestrate stuff like that within Python. The multiprocessing module.

[giantrobot → pookha]

You could have done this with the multiprocessing module. It interfaces like the threading library but it's launching new processes for each "thread". I've done a shitload of image processing in Python with it. You'll end up with a 100% Python version of what you did.

[ska → pookha]

Multiprocessing does basically the same thing, but in python.