A message queue stores, well, messages, so it is really good for events. But I also need to persist state machines that run for months. They must be queryable and so on. A db is good at this. On the other hand, I've had a very unpleasant experience with the database-as-IPC-channel and do not wish to ever go near that again. So using a rdbms to persist state and a message queue for the events seems like the conservative approach. Am I overlooking something obvious here?
Not at all, just wondering about your thinking. Note that I think using a database for IPC is a bad idea if you're actually using tables and such to coordinate; Postgres' listen/notify IPC is much better.
That said, it has issues, as pointed out elsewhere in this thread; when you notify, all listeners wake up, which is not what you want with a one-job-per-worker dispatch system.
If you abstracted the listening to a central job manager app, however, you could use "listen" to wake up just the job manager, and then use that wake-up to dispatch the next job to an available worker.
