This is surprisingly non-shitty by Google. I must admit that I didn't know that before. Can you limit such a passcode to just IMAP/SMTP, or can it be used to log in to other parts of Google?
This passcode is inherently limited to the service it bound to (IMAP or POP3), that's the whole point: don't expose your account password to something which only needs a finer-grained access.
You're directly contradicting your sibling comment. I guess I'll experiment with this in the coming days, although I'm a little worried tinkering too much will just completely lock me out of my account.