This isn't true even with a organized, disciplined team. What happens is the responsibilities of get divided out to other people on the team. Usually not the entire team, so some people are stuck with unacknowledged overhead.
Unacknowledged overhead happens even with a PM present. Usually it's work that is absolutely critical to getting a project done but a PM would tell you it's a waste of time if you brought it up with them.
