Forcing everyone to use e-money only has a few drawbacks. There will always be things that you don't want to be associated with - even if prostitution is legal in your country, if you are married you would prefer this transaction to be untraceable, I suppose.
Another possible drawback is that once money is only available through "controlled" sources, you could find yourself subject to redlining (see: http://www.engadget.com/2015/12/02/paypal-square-and-big-ban...).
So while I find e-money convenient and practical, I do not think that society would be "better" if physical money went away.
So while I find e-money convenient and practical, I do not think that society would be "better" if physical money went away.