you can't shrink your head. The smallest version is a pair of glasses. There was a privacy concern when people started wearing Google glasses in public if I remember correctly. The norm has to be respected until the norm changes.
The problem with Google Glasses was that they looked basically like glasses (being AR), in many ways other than size - starting with the fact that they were transparent. A VR headset, even if it's equally compact, won't look like glasses - certainly not if the designers don't want it to.