You can put it on your CV, although it is not important. Being a reviewer counts as "service" for your annual performance reviews, which is slightly more important. But most importantly, nobody wants to waste time reviewing mediocre papers. Everyone wants to review the best papers, from which they can learn the most while still fulfilling their reviewer duties.
Yes, unfortunately, it is almost more important to reject the bad papers. The good papers will be published regardless of the reviewer. It takes more work when I get a bad paper, especially when it comes from someone famous, because I have to make a convincing case for rejection. I've even gotten the same paper multiple times from different journals (after rejection an resubmission). It's hard work to prevent flawed science from being published.
