I'd say most of us see "the other half" as the rich, because we see ourselves as poor, relatively speaking [0]. I think Riis titled his book to give the rich an idea how bad things were for others, to try and draw some empathy from them.
0 - of course, wealth distribution isn't 50/50 today (and probably never was)
