This is a great idea. You could use public key cryptography too, so that the system adding emails to the db doesn't need the private key.
3rd party mail sending services could support this by generating a keypair on their systems, and only giving you the public half. When you make an API request to send an email, you provide only the encrypted version of the address.
Edit: The hashing is an issue. It's too easy to build a wordlist of possible addresses, to crack the hash. I think this can only work if you drop the hash column, and instead require users to log in using a username.
The hashing is an issue but you need to identify the user somehow when you do things like password resets.
The alternative is to handle everything by a username and password resets also use the username (which would be fine, worst case you get spammed by PW reset mails).
Though of course you can also combat this by making the hash particularly expensive and salt it. Simply take a SHA3-512 of the email address a few thousand times, take the first 12 bits and use that to identify a set of 4096 records. Now the full email is simply an application of Blake2sp, which you calculate in parallel for all 4000 records.
Adjust the 12-bit barrier so that it represents a decent sized chunk of users, lower would mean less load on the login service, higher would mean better anonymity. Instead of SHA3-512 you could also use a bloom filter to find out if a set of records contains the email or not, with the added bonus of being probabilistic.
You could also ditch Blake2sp for a simple round of salted SHA3-512. The fact that you salted it makes dictionary search insanely annoying already.
3rd party mail sending services could support this by generating a keypair on their systems, and only giving you the public half. When you make an API request to send an email, you provide only the encrypted version of the address.
Edit: The hashing is an issue. It's too easy to build a wordlist of possible addresses, to crack the hash. I think this can only work if you drop the hash column, and instead require users to log in using a username.