I'm running GrapheneOS on a 4a right now and it's smooth like butter - maybe you needed to wait for a few updates. The camera has improved a lot as well but is still not close to the stock google camera.
It seems like what you're looking for is CalixOS + microG.
It seems like what you're looking for is CalixOS + microG.