I think it should probably company policy at a financial institution not to allow someone in a different control on a handheld device to access the servers.
I use layered security (in the sense of servers) with a password lifetime of 10 days. I deal with servers daily and the issue of compromise has been twice in on firewall containers/servers which was fine as the systems in place found it, found the issue with the net facing software and I fixed it. Practicality doesn't mean no security.
Can you require keys AND passwords? I haven't been able to figure out how to get that to function - if passwords are allowed, it lets you in with or without a key, from what I can tell.
I'd be happy to be wrong though!