Hacker News new | past | comments | ask | show | jobs | submit login

You can have remote workers with parallel (not sure if you can do that with xargs or not).

    parallel --progress --wd '.' -S '4/fourworkers@blehworker.org,8/eightworkers@eightworkers.org,2/:'  "./complicated_command --param1={1} --param2={2} ::: $(some_shellcommand_to_generate_param1_values.sh) ::: $(some_shellcommand_to_generate_param2_values.sh)
It's very flexible, become my go-to for speeding up parallelizable work. Next step up would be Celery or Spark or something.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: