I'm English and I've worked in the Isle of Man - and getting a work permit was just a formality. You are also ignoring the money the IoM hands over to the Treasury to support the defence of the British Isles. The "neutral" Republic of Ireland genuinely free-rides on UK defence (and historically actively worked to undermine Northern Ireland security) and so would be a far better target for your complaints.
GP post's complaints don't apply in Ireland: A British citizen from Liverpool can arrive on the boat in Dublin and start working that afternoon with no permits. Looks like you're throwing in a red herring.
Only for the EU. As a Manxman you have a nice one-way street of benefits working in the UK...
>Manx people, as British Citizens, may travel and work freely in the United Kingdom.
However myself as a British citizen cannot take a ferry from Liverpool to the IoM and start work. But the reverse is entirely possible for yourself.