A contract implies a meeting of minds. I do not accept any system which has systematically denied me representation my entire adult life as legitimate in any way.
A system imposed on people which denies them representation has no claim to be considered democratic.
All representative systems are somewhat undemocratic but the point of them is to pick someone who's we can agree is legitimately the winner of the contest - not to deadlock trying to pick a winner to a better standard.
The primary goal isn't to represent, but to reduce misrepresentation to the point where we can literally live with it.
To me it's only illegitimate if this is coupled with unreasonable attempts to keep us from improving the system as technology and knowledge give us the means.
The social contract isn't to be "only be ruled by a strict majority" but to agree to be ruled by the party who wins by any workable system.