You're in a confined space, breathing the same air as other people. Also, given the frequent traffic of elevators, consider what you may breath in from previous occupants. This sounds risky to me. I hate touching elevator buttons during normal times. I'd wear masks and gloves.
