Having always been on AWS and using Glue Spark Jobs for my jobs, I've never felt... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

mkw5053 on Oct 8, 2021 | parent | context | favorite | on: ETL Pipelines with Airflow: The Good, the Bad and ...

Having always been on AWS and using Glue Spark Jobs for my jobs, I've never felt any benefit of using Airflow for orchestration over Glue Workflows. I can understand some people not wanting to deal with vendor lock-in. I'm curious what others opinions are.

higeorge13 on Oct 8, 2021 [–]

For glue jobs you probably don’t need something more. However step functions is the norm for creating workflows in aws nowadays.

Join us for AI Startup School this June 16-17 in San Francisco!
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact