People have been conditioned to do so. There's no intrinsic reason why property rights should be protected to a greater degree than workers' rights, but libertarianism promotes awareness of one and not the other. It's kind of a circular reasoning, but useful as an observation.
That's an overly broad and inaccurate depiction of libertarianism on the issue. A considerable segment of libertarianism advocates such measures as land value taxation, which places it directly at odds with the land owning class.
