Imho "source" is an anti-feature. I think instead of "source", you always should create a subshell. It properly encapsulates your modified environment and avoids leakage and confusion. Exiting an environment is a straighforward exit/<ctrl>-D like with any shell. You do not need to remember a special "deactivate" command like with virtualenv.
I swear by subshells. They solve so many problems. People object that they don't let you do environment autoloading on `cd`, though (which I also hate, so... Yeah).
Obviously, virtualenv and others disagree. ;)