My major concern is that there is a huge difference between the actual usefulness of unix signals, that is mostly limited to killing processes or signaling tiny bits of information (like when you send a signal to reload a configuration), and their semantical complexity.
I must admit it is not trivial to replace this system with another equivalent without changing a lot of how Unix works. For instance signals are used in order to interrupt a program: if you don't make this trappable (like kill -9) then programs can't recover or prepare in any way before quitting. An alternative can be to just allow a signal handler to fire that will never return.
From this point of view you could even just have two signals, with the only goal of stopping processes, one in an hard way (SIGKILL) and one in a soft way (SIGINT), firing a signal handler (but the process will terminate when the handler returns). This way you don't need handling the interruption of syscalls nor writing "safe" signal handlers. As you said signals are interrupts. If the interrupt handler can never return a lot of problems disappeared.
All the other tasks currently performed with signals should use a more general and saner message bus, with select(2)able file-alike interface.
Plan 9 uses a system called notes in place of Unix signals. It's pretty much the same idea, but instead of sending an integer you can send arbitrary strings by writing to /proc/n/note. Non-trappable messages are sent to /proc/n/ctl.
This is a more flexible system than Unix integer signals, but in practice the extra flexibility isn't often used, as a process can simply mount a ctl file into the namespace or post a pipe in /srv.
I always thought it'd be cool if all programs implemented external apis this way. Kind of like how applescript works, but, for a mail program, for instance, write "Hacker News" to /proc/n/query and it'd return the data. Or write 'message 1' to /proc/n/open and it'd open or return it.
You wouldn't put those sorts of things in /proc, but that's basically how things work. And since it's a filesystem, you can export your API effortlessly over the network.
I must admit it is not trivial to replace this system with another equivalent without changing a lot of how Unix works. For instance signals are used in order to interrupt a program: if you don't make this trappable (like kill -9) then programs can't recover or prepare in any way before quitting. An alternative can be to just allow a signal handler to fire that will never return.
From this point of view you could even just have two signals, with the only goal of stopping processes, one in an hard way (SIGKILL) and one in a soft way (SIGINT), firing a signal handler (but the process will terminate when the handler returns). This way you don't need handling the interruption of syscalls nor writing "safe" signal handlers. As you said signals are interrupts. If the interrupt handler can never return a lot of problems disappeared.
All the other tasks currently performed with signals should use a more general and saner message bus, with select(2)able file-alike interface.