The perception could be because the traditionally poorer American minority groups are not as common in San Francisco: there are not too many Latinos (14%) or especially African-Americans (7%) compared to other major American cities. Not sure if that's for cultural reasons, or just because SF is expensive, so there aren't as many poor people of any color as in many cities (e.g. compare SF's under-$30k-income population to Chicago's). Might be related to the SF versus East-Bay de-facto segregation as well; it seems most of the Bay Area's Latino and Black population lives in the East Bay (e.g. Oakland is 30% Black and 25% Latino; Richmond is 36% Black and 27% Latino).
