But it's also very important to understand why machine learning systems do this. If you take race neutral data, and run virtually any machine learning system on it, you'll get a race neutral output. I.e., if a $40k/year black person in 10001 is equally likely to default as a $40k/year white person in 10002, then the algorithm can be expected to give them equal credit scores.
In the event that an algorithm choose black zip codes for extra penalties, it's because the algorithm has deduced that stereotypes are correct - the bank will make more money not lending to blacks at given rates, because something directly correlated with race and not captured by fields like income/past defaults/etc is predictive.
Having consulted for folks doing lending, I'll mention an insanely difficult $100B startup idea here: build an accurate algorithm that exhibits no disparate impact. The amount of money you'll save banks is staggering.
True, and then the question becomes, is the stereotypes correct, or is it that the credit score is then propagating a system that enforces this outcome.
The stereotype (That predominantly black households in a poor area code are poorer then the predominantly white households in a rich area code) can be proven correct by simple demographics, and the fact that household wealth is correlated with credit score.
The problem is that there's a positive feedback loop at play, here.
The bank already knows their assets and income. The question is if an equally poor and educated white is just as likely to repay a loan as an equally poor and educated black. I imagine most of the difference would go away. Unless you really believe black people are inherently less likely to pay back loans, all else equal.
If you can come up with an algorithm that reproduces this conclusion - with accuracy even remotely close to the "racist" ones - banks will cross heaven and earth to pay you $billions.
I do in fact have an algorithm to remove racism from models, but I doubt its worth "$billions". The whole point of my argument is that it shouldn't be necessary. Surely you don't really believe racist stereotypes are true?
I for one of course do not believe that people from other regions/continents are somehow inherently worse or better.
But I do believe that certain elements of different cultures and different ways of social upbringing can have a lasting positive or negative effect on a person. (although who am I to judge what is positive or negative? I try not to do this, I just see differences)
If these things didn't affect how we later as adults view the world, interact with others and respond to different types of challenges in our life then you could expect that basically everyone around the world would have the same moral value system and beliefs about almost everything.
I do, in fact. There is extensive research supporting the fact that many (though not all) are accurate. The typical racist stereotype is about twice as likely to replicate as the typical sociology paper.
I didn't say that simply removing "racism" (by which I assume you mean disparate impact) is worth billions. I said doing so with the same accuracy as models which are "racist" is worth billions. Obviously you can do anything you want if you give up accuracy.
Why do you believe they are false? Simply because it's socially unacceptable to think otherwise?
I'm not sure whether or not the stereotype is true. The point is you are forced to acknowledge a pretty unpopular belief to defend this idea. And that the kind of people super concerned about disparate impact are usually not the kind of people that believe that. And I think that if you do acknowledge there are differences, the argument that it's unfair and wrong is much less strong.
In any case the link you posted earlier suggested it could be explained by different rates of numeracy and literacy. Which sound like obvious proxies for IQ. If you gave people an IQ test, or at least literacy or numeracy test, this would probably eliminate any racist bias.
> I said doing so with the same accuracy as models which are "racist" is worth billions.
You necessarily lose some accuracy if you really think race is predictive. But you need not lose all of it. The current methods for removing bias just fuzz features that correlate with race. But there is a way to make sure a model doesn't learn to use features just as a proxy for race, but instead to predict how much they matter independent of race.
> Simply because it's socially unacceptable to think otherwise?
Please stop insinuating this at people in HN comments. You've done it frequently, and it's rude. (That goes for "simply because of your mood affiliation?", etc., too.)
Dang, I'm really confused here. As Houshalter points out, he was deliberately making an argument based on social unacceptability of my beliefs. Searching for the phrase on hn.algolia.com, I've used the term 5 times on HN ever, never in the manner you imply.
I'm also confused about my use of the term "mood affiliation". Searching my use of the term, most of the time I use it to refer to an external source with a data table/graph that supports a claim I make, but a tone which contradicts mine. For example, I might cite Piketty's book which claims the best way to grow the economy is to funnel money to the rich (this follows directly from his claims that rich people have higher r than non-rich people). What's the dang-approved way to point this out?
In the rare occasion I use it to refer to a comment on HN, I'm usually asking whether I or another party are disagreeing merely at DH2 Tone (as per Paul Graham's levels of disagreement) or some higher level. What's the dang-approved way to ask this?
> usually asking whether I or another party are disagreeing merely at DH2 Tone (as per Paul Graham's levels of disagreement) or some higher level. What's the dang-approved way to ask this?
That seems like a good way to ask it right there.
Possibly I misread you in this case (possibly even in every case!) but my sense is frequently that you are not asking someone a question so much as implying that they have no serious concern, only emotional baggage that they're irrationally unwilling to let go of. That's a form of derision, and it doesn't lead to better arguments.
But if I've gotten you completely wrong, I'd be happy to see it and apologize.
As the person he was replying to, that statement wasn't rude in context. The belief being discussed really is socially unacceptable. My argument in fact, relied on that.
In the event that an algorithm choose black zip codes for extra penalties, it's because the algorithm has deduced that stereotypes are correct - the bank will make more money not lending to blacks at given rates, because something directly correlated with race and not captured by fields like income/past defaults/etc is predictive.
I discuss this in greater detail here, taking linear regression as a toy example: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
Having consulted for folks doing lending, I'll mention an insanely difficult $100B startup idea here: build an accurate algorithm that exhibits no disparate impact. The amount of money you'll save banks is staggering.