The tl;dr is: because there exists systemic disadvantage-ous-ness clearly caused by, and maintained along lines of race. It's not just The Past™ that was racist to damaging degrees, Now™ is too.
All people in bad circumstances beyond their control deserve ways out. There are tons and tons and tons of subsidies and whatnot targeting individual reasons, like poverty (people think you're poor? you're less likely to get a good-paying job, and more likely to get one that abuses you). Race is just one more, and it's particularly effective at leveling the field because it has extremely strong correlations with (and very clearly a cause in) tons of them.
But with affirmative action it's about race and not being poor. It's not a measure that looks at household income, but a measure that looks at skin color. Poor white Bobby from a trailer park in alabama is financially fucked, and noone cares about him, because he's white.
All people in bad circumstances beyond their control deserve ways out. There are tons and tons and tons of subsidies and whatnot targeting individual reasons, like poverty (people think you're poor? you're less likely to get a good-paying job, and more likely to get one that abuses you). Race is just one more, and it's particularly effective at leveling the field because it has extremely strong correlations with (and very clearly a cause in) tons of them.