I am working on file support right now as a destination to begin with. I believe I should get local files as well as S3-compatible sources going by tonight.
Reading the sources in parallel is an interesting idea, I'll definitely take a look at it. ingestr supports incremental loads by a partitioned column, but there's no parallelized partition reading at the moment.
Reading the sources in parallel is an interesting idea, I'll definitely take a look at it. ingestr supports incremental loads by a partitioned column, but there's no parallelized partition reading at the moment.
Thanks a lot for your comment!