I think this rule is just wrong. Some mathematicians come from privilege, some do not. Even for the ones who do, it's rare that they come from _great_ privilege for the time. They are the children of mayors, not kings -- professors and engineers, not the financial aristocracy.
Now it seems you're moving the goalposts on what counts as privilege. Do you have any quantitatively backed reason to believe Fields medallists are more likely to come from the financial aristocracy than say Abel prize winners?
I don’t have access to a biography of all the Field’s medalists. I am simply pointing out that there is a wide distribution of financial privilege in a list of famous mathematicians that I did not choose. I would argue this is unexceptional, but I point it out because I have trouble with the hypothesis that one must come from privilege to get anywhere in math. If anything I should be asking for the quantitative proof of the hypothesis.