The private key is on a server you have physical and logical control over. The public key, however, is potentially on millions of consumer devices – some hacker reading out the public key is a question of when, not if.
Asymmetric key exchange is done with a private key on both end points. For things like HTTPS the “client” key is ephemeral. But if you are using the keys for authenticated communication, which I think is what this thread is about, both keys are vital (think: client-side certs).
