I think a big portion of those people come from the fact that it's essentially impossible to verify what each and every piece of telemetry does. And ultimately, you as a user don't directly gain anything from enabling it / leaving it enabled, so it's perfectly reasonable to stay ignorant in this case.
