While adding backdoors to encryption is a bad idea my gut sys decentralized reputation is just a bad. Most here know the Black Mirror episode "Nosedive". But it's worse than that, people on different sides of the political spectrum will work to "cancel" people of the other side by working to lower their rep via crowdsourcing. 4chan type groups will do the same for "the lulz", I can even imagine the ransomware people trying to find a way to make bots and "pay us or we destroy your rep"
Totally agreed that badly designed rep systems can rapidly descend into Black Mirror territory. In Nosedive for instance the score you get is absolute, while the thing we’re proposing here is entirely relative and subjective. The idea is to empower users to maintain their own view of the world. If a voting ring of idiots conspire to try to trash someone’s reputation... you’d filter out the voting ring from your reputation feeds; it should stand out very clearly. That goes whether it’s bots or humans behaving like bots. Alternatively, you could choose to hang out with that tribe and believe their rep data if you so desired.
We’ve been wargaming through all the various ways this could go horribly wrong (and have a few fun scenarii off the back of it - look for the GPT-3 example below), but on balance it feels a lot better than the Black Mirror episode where encryption is fatally weakened...
Overloading the word 'censorship' is going to get a bit confusing here ;)
"Censorship", meaning: "malicious server or ISP silently blocks or withholds traffic from you" is a risk in Matrix today, completely independently of the reputation stuff being discussed here. The mitigation is to get rid of servers (and even ISPs), as per https://matrix.org/blog/2020/06/02/introducing-p-2-p-matrix/
"Censorship", meaning: "your server admin subscribed to a blocklist of child abuse content published by someone like iwf.org.uk" would stand out to users by the server publishing the names of the blocklists that their server admin has deployed. I'd call this server-side filtering or something instead, given the filters are visible. If the server admin withholds info about their filters, then you're back in the traditional sense of censorship from the prior paragraph.
Are the blocklists public? I thought these schemes involved uploading files to a "trusted third party" like Microsoft, and for them to decide if the files are lawful?
Unfortunately, I don't think that'll work. For it to work, you would need every actor to play along, otherwise someone could just move to Tor to do the exact same thing. I don't think it's possible to eliminate what they want to eliminate, and they will only push for more and more.
My suspicion is that this system is a cop-out, like when Google blames their screw-ups on "the algorithm".
There probably has been a recent surge of right-wing users and, the Matrix administrators, which probably lean left-wing, have seen that surge as a problem and have created the reputation system as a way to get rid of those users or, at least, to give them the hypothetical yellow badge. And if someone says "censorship", they will blame it on the "downvotes" those users got.
