That's basically what TOTP/HOTP authentication tokens are, which many sites (including Google, AWS, Github) etc use for 2FA - https://en.wikipedia.org/wiki/Google_Authenticator.
When you set it up, the service provider creates an 80 bit secret key, which you enter into your local device (or some implementations create a QR code) and then whenever you log in you need to provide a 1-time password from the app.
I'm aware of 2FA using (T|H)OTP, my thought was that a GPG/SSH key can be stored in a secure and yet reasonably easy to use way, effectively offline (i.e. add a passphrase and store it on a USB key or similar).
With a 2FA code, you either a) use the same code they use for regular logins, or b) require them to find a way to securely store the (T|H)OTP secret and then add that information to a 2FA app when they want to do a password reset.
I realise the pubkey concept is more than most people would bother with (or even be able to get through on their own), and I think the first 2FA option is definitely better than no extra security at all on password resets, but my thought was about increased security for those who are particularly paranoid/security conscious.
