That's pretty much what Jamulus does: sound is mixed server-side, so you have a single point of reference (can be click track, can be drummer, can be sequencer...). You can even get a multi-track recording saved on the server so you can then adjust timings and give it a proper mix afterwards.
