Im sure plenty of people feel that way, but they have no reason to go around telling everyone that, especially when many other people will argue against them and admonish them for it.
Do you have a reason why you're pretty sure plenty of people feel that way? The causal mechanism you're proposing for their silence—chiming in and getting admonished for it—would lead evidence that I don't see. Unless you're suggesting that people just assume they'll get chided?