This feels akin to counting all time in gmail as "work" regardless of what one is reading/writing. The most dangerous distractions are those which disguise their presence.
The mental burden of task switching is about changing mindsets, not applications. The damage here is equal whether you're looking for an album in vim or a gui.
So I suppose that you don't want to try the same programmer's Vmail[1] - a Vi-interface for Gmail? (Oh, or even better Twim[2] - a Vi-interface for Twitter!)
More seriously, I thought of it as (1) a cool hack and (2) akin to Zawinski's law[3] about the way that software expands to consume more tasks. More serious Vim hackers might object that it amounts to the Emacs-ification of Vim. (Given the number of plugins to add calendars, todo lists and Org-mode to Vim, I think that battle may be lost.)
I've been using mutt with gmail. It drops into vim when it's time to compose a message. I've never heard of vmail. It sounds interesting, but I would personally recommend mutt over vmail. mutt is quite vimish.
You don't have to store your password in plain text just because it gives you the opportunity to do so - I use mutt without a stored password for IMAP. It means that I have to enter my password between one and three times per day, but that's a minor inconvenience to me.
While I tend to agree in general, I've actually found pianobar much less distracting than the Pandora web interface.
F5 (my hotkey for Terminal.app), n, Alt+Tab can happen without me really thinking about it, whereas the web interface requires firing up the browser, locating the "Next" button and clicking it.
So I guess I can see this being an improvement if all you're looking to do is skip to the next song. If, as you say, you browse through your albums, etc, then I imagine this would be just as distracting.
I agree with you that the claim that this is some sort of "productivity enhancement" is a bit weak, but as an actual feature, it is pretty impressive. The productivity side of it is immaterial, IMO.
The mental burden of task switching is about changing mindsets, not applications. The damage here is equal whether you're looking for an album in vim or a gui.