I've had that concern as well but have not found it to be a problem in practice. For one thing, it's not hard to go back to default vim if you're on a different system. For another, it's usually not that hard to copy your .vimrc over if you're going to stay awhile.
