If it's dangerously unknown to the contractor, chances are inhabitants won't know better. The contractor only exposed for a few hours, inhabitants live there.
Average construction zone concentrations of ACM during demolition hover around 1fcc. It can be higher for some specific cases (zonolite or high concentration asbestos insulation) but for general things like contaminated drywall, duct wrapping etc it is generally low. This is during actual demolition of ACM mind you, not just disturbing a little drywall here and there.
When disturbance is completed, concentrations fall of with the air exchange rate and clear usually with 24-48 hours back to background levels.
Diluting that by the volume of the entire building not under construction would generally put that at or below the 0.1fcc OSHA limit without PP. The OSHA limits assume 1 extra cancer death/300 workers for someone with maximum exposure limits for 8 hours per day, 250 days / year for 30 years.
That is to say… A contractor spending a day kicking up asbestos results in 1/7500th ish of the dose required to cause an excess of 1/300 deaths.
And this assumes that acute low exposures hold the same linear dose response rate as high dosage chronic exposure. That is a controversial assumption in the literature.
My information is based on reading over 50 asbestos related journal articles, EPA and OSHA policy documents etc.
From all the informarion available, it’s just not that big a deal for the average building occupant.