I'm not saying it wasn't legitimate or legal, just that _most_ of us did not vote for it, which is why I felt "bone headed people" was too much of a generalisation.
With power comes responsibility. The flip side of suffrage is responsibility. Abstaining from (or casting a blank ballot in) an election or referendum is, practically, dividing your vote amongst those who show up.
(Someone who is not eligible to vote is not really one of a country's people. They're its visitors, children, felons, what have you.)
It's arguably more significant in this case as those that didn't vote were around 75% in favour of remain. It's not common to hear of such a differential between the intentions of those who did not vote compared to those who did.
Had everyone voted, remain would have won by a large margin.
There is a key difference betwen "this is the will of the people" and "this is the will of the people that we allowed to vote, and who found the time to make it along on the day".
People aged 16-18 and people living abroad are two groups that had no option but to sit at home and are both considered anti-Brexit.
That argument sets no principle for declaring a government illegitimate, so you should really have started the second paragraph with “By my hostile reimagining of that line of reasoning...”
By that line of reasoning, every government we've had in my lifetime has been illegitimate, elected only by a tiny minority of the population.