Isn't that just a psychological thing? I mean, if people don't wash their hands after using the bathroom they might be less likely to wash their hands in general, but they're still using other doors, kitchen appliances, papers and (depending on culture) you shake hands with them in the morning. There's no way to avoid that while not looking socially awkward, just wash your own hands more.
Also, don't you remove some of that bacteria left on the door handle by touching it yourself? The amount of bacteria on bathroom door handles should remain rather constant given that you take some away as you leave some of your own, also door handle not having favourable replication conditions for germs.
Well I haven't done any studies on it but I'd assume that the number of people going to the toilet and then immediately touching the way-out door handle without washing their hands is probably higher than the number of people going to the toilet outside the bathroom and then touching the door handle on the way in...
Not to mention that bathroom door handles are pretty good conditions for germs: warm (indoors), damp (from people who haven't dried properly), lots of new germs from people who haven't washed.
Also, don't you remove some of that bacteria left on the door handle by touching it yourself? The amount of bacteria on bathroom door handles should remain rather constant given that you take some away as you leave some of your own, also door handle not having favourable replication conditions for germs.