Hacker News new | past | comments | ask | show | jobs | submit login

You can use messaging to distribute the work, but let all workers access the same RDBMS; that way you pretty much get the same ACID properties you are used to.



We are currently building a system that does this, but it feels inefficent.

First, write the job to the DB, then put a message on the queue to notify the job processor(s) that there's work to be done. The processor then updates the DB to indicate that the work is done.

The goal is, of course, to prevent polling the DB via repeated queries for jobs. But wedging in an entirely new layer/API/messaging server just feels like overkill for the simple task of initiating a job.


There are two answers I have to that.

First, scaling often involves some things that feel unnecessary or inefficient

Second, once you have a messaging system, you find lots of good use cases for it (other kinds of notifications, logs, statistics, integration with other systems, ...).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: