Or store your keys in a different user. I personally use have the users dotancohen and dotancohens (trailing s for Secure) on my laptop. I simply su into dotancohens and then from there SSH into various servers. The /home/dotancohens/ directory has 0700 permissions.