They work as a pair. If you access from a different machine you need the private key on that machine. You can't just copy the public key over to that machine and expect it to work.
It depends on what you are doing. You can do public key operations without the private key (for example signature verification). You can do private key operations without the public key (for example signing). In practice the certificate is always available near the private key, but the reverse is usually not true -- there are many cases where you will have the certificate but not the private key.
