That's actually what Dr Fauci initially talked about with regard to the general public. With how masks are worn and used in practice, he was probably right.
Fauci: There’s no reason to be walking around with a mask. When you’re in the middle of an outbreak, wearing a mask might make people feel a little bit better and it might even block a droplet, but it’s not providing the perfect protection that people think that it is. And, often, there are unintended consequences — people keep fiddling with the mask and they keep touching their face.
LaPook: And can you get some schmutz, sort of staying inside there?
Fauci: Of course, of course. But, when you think masks, you should think of health care providers needing them and people who are ill.
> "Well, the reason for [not telling people to wear masks in March, 2020] is that we were concerned the public health community, and many people were saying this, were concerned that it was at a time when personal protective equipment, including the N95 masks and the surgical masks, were in very short supply. And we wanted to make sure that the people namely, the health care workers, who were brave enough to put themselves in a harm way, to take care of people who you know were infected with the coronavirus and the danger of them getting infected."
Laudable as his intentions may be, and while I doubt he makes it a habit, he lied.
It could be that cost/actual benefit looks worse when you factor poor usage.