It wasn't that long ago when there were only 4 unique keys in the world and a thief had a skeleton key that would replace them all. In those days nobody locked their door because there was no point. As lock technology got better people in places of crime installed good locks and told their friends to do the same. Most people have no need to lock their door, but nobody knows if they are the exception.
With all respect, I call bullshit to your "4 unique keys" theory. Locks have a long and varied history all around the world. There is no way a single skeleton key would even fit into an Arabic, Chinese, Indian, Venetian, Swiss, or German lock.
For practical purposes you are wrong. In any given city 4 door locks is about right. There are padlocks and other types of lock, but the subject here is door locks in one location so4 is close enough and was the truth in the US where my grandparents grew up.