On: Just use Postgres for everything

[scrollaway]

I'm just about there and agree in general.

Currently integrating Procrastinate (https://procrastinate.readthedocs.io/en/stable/) to use Postgres as a job queue in a Django API backend.

Dropping dependencies is very nice especially when dealing with an MVP / small apps.

[phoebefactorial → scrollaway]

I do Django freelancing and have had a fair amount of small clients with relatively few users. Adding workers and a message broker adds a lot of complexity over just a single Django server. I even had the thought to build something like this but looks like someone already did! Thanks for sharing this, looks like a great tool to know about.

[whatever1 → scrollaway]

Why not the battle tested celery?

[llIIllIIllIIl → whatever1]

Battle tested Celery has dropped the ball so many times during operations and updates, that I would be very uncomfortable giving that recommendation.

[simonw → whatever1]

Here's a really good article about why you might want to involve your database in queues (short version: transactions): https://brandur.org/job-drain

[scrollaway → whatever1]

See article?

But also, Celery has an awful DX. I think the question is more, why do I need celery when Postgres can do the job itself?