I've walked through the downtown of most major cities in the US, day and night, and the only one I've ever genuinely felt unsafe in is SF.
Being "street smart" and constantly aware of your surroundings/situation will get you through most cities just fine but there is something utterly unpredictable about the street scene in SF that makes this strategy unworkable. It's basically one step above total Mad Max Anarchy.