I find the whole "you go" thing annoying too. Someone in a vehicle can simply modulate their acceleration pedal and be out of there in a few seconds. Getting out of an intersection with a bike isn't as quick.
In states where right of way is codified and well defined in law, generally, "you go" is a ticketable offense (you are instructing someone to break the law, technically, by telling someone else to violate right of way). Additionally, as I once saw it noted, "By not taking your right of way, you are not being courteous, you're being unpredictable."
I'm not sure I follow your logic. If the other vehicle has stopped and the driver is requesting you to go, they are not attempting to travel into your path, and so there is no conflict of right of way. Right of way laws aren't written to tell drivers when they must go, but rather to tell drivers when they must yield.
You don't know that they're not attempting to travel into your path. Once they ignored the rules of the road and became unpredictable, you need to treat them as possibly doing the stupidest thing for the given moment.
Many a time I've come up to a stop right after someone else, and they start waving me on. But almost invariably, unless I immediately go (basically without even waiting for the 'go on' wave) they will wave me through, and then start driving through the intersection.
The point is these 'go on' people _are_ ignoring the laws about when a driver must yield. They're trying to tell a driver that must yield that its okay if they don't.
