Here we are, deep down in the dystopia of Automatic Content Classification by robots.
Far less important of an example, but I was just put in "Facebook jail" for 24 hours for posting a picture of my son at the beach in his bathing suit with no shirt. Y'know, as one does at the beach. I can only assume it's because my son has long hair and the Convolutional Neural Network or whatever decided he was a girl and therefore I'm a pervert.
Sadly it was the "appeal" I submitted that got me blocked. Presumably by a "human", but who knows.
Before I appealed they were simply going to not show the picture. Appealing got me in "trouble." That might be even worse than the original misclassification. On top of that, if I was actually a "community standards" violator who posted potential child exploitation imagery, I'm not sure how a 24-hour ban on activity on Facebook is of any use, either? Except I'm terrified to imagine a world where Meta might have called police on me based on the output of a neural network image classification.
Others have said but I'll say it again: This kind of business doesn't scale ethically. You can't have billions of people on a bulletin board. It doesn't work. Moderation is essential to modern communication. But you can't do moderation automatically or at scale and in a universal way.
Very dark patterns emerge the moment you go FAANG scale and toss algorithms tuning for advertising and "engagement" into the mix, and attempt to do so with the help of computers.
"Sad" as it is, we will need to "retreat" back into smaller forums and BBSs where communities self-police.
Facebook has infiltrated so many aspects of society. Want to interact with the parents from the local school your kids go to? You have to do that on Facebook. Event announcements? Keeping in touch with your distant aunt? Facebook.
If something like Facebook is really a universal utility, it will have to be put under public administration; like the post office. But clearly, that isn't going to happen and would have other problems.
I am going to have to find some other way to engage with old friends and family.
Frank Herbert had some intuition with his whole "Butlerian Jihad" thing.
If you are only using Facebook for its original purpose of keeping up with old friends and family, then it doesn't actually have the scaling problem: if one of your friends or someone in your family starts posting a ton of racist bullshit you either confront them about it or drop them as a friend (preferably bilaterally), and either result is actually better for society than having Facebook attempt to present a skewed view of them that tries to just pretend they aren't posting all of the horrible stuff in the first place (whether by blocking it from being posted, quickly removing it once posted, or running some complex ranking algorithm that does a good job of hiding it).
It is only when you start having strangers talking to strangers that you run into a need for moderation, and even there you should be able to scale by sharding and punting the problem to others: if you have a group--similar to a real-world club--the moderation is on you, as the issues in your community shouldn't leak to people who haven't joined your community (and if people leave your community because you fail, all the better). The only real issue is that Facebook wants to--for the increased engagement, and thereby ad revenue--run a ton of recommendation algorithms that shove content from people you have no affiliation to in your face constantly (which one might notice should already be considered antithetical to the design of a social network) which leads to a ton of stranger-to-stranger interactions that are entirely "on Facebook" to ensure are clean.
And yet here I was, posting a picture of my kids at the beach so my mom and friends could see them, and I ended up banned from participating on Facebook for 24 hours and accused of violating community decency.
Yeah... so I honestly consider this to be a separate (also horrible, for avoidance of doubt) problem than "moderation"? Like, Apple and Google aren't exactly having a moderation problem when it comes to attempts to curtail CSAM stored on their photos platforms (which has led to their automated flagging systems and then need to scale to the entire world of users appeals and escalation, and then the subsequent concern about "are they going to call the police on me when I fail my appeal?!")... it is more of an attempt to deal with awkward regulatory "think of the children" overreach and hostile American law enforcement. Facebook seems to me to be doing the same thing here and failing.
Imagine a world where social networks were built by people who simply didn't care about "engagement" at all and weren't being motivated by ad revenue... I think you could design an end-to-end encrypted version of the system where no-one except your friends--and certainly not the network operators--even knew what you were posting in the first place and they would hopefully be able to avoid installing client-side filters for CSAM (but, with stuff like FOSTA and SESTA, maybe not?). This model should even work, I'd think, for Twitter/Instagram-like broadcast models (though the legal implications of the well-known "secret" key being published and accessible to the network might lead to various problems; you might have to go fully-decentralized).
> using Facebook for its original purpose of keeping up with old friends and family
Not that Facebook lets you do that any more. I was off of FB for a while and recently rejoined because my community and my kid's schools only use FB for communications now. I'm connected to a very small circle of actual friends and family, but I still get daily political memes in my feed (politics I vehemently disagree with as well) no matter how many times I try to block them.
The biggest spam I get from Facebook ads itself. I live in Switzerland and I am BOMBARDED with financial frauds, scams and ponzi like schemes served directly from their ads. I checked why in their system and got answer "primary location: Switzerland and male 25 - 35 years old". I tried to report, block them all but facebook support always if replied at all, it was saying all in line with their policies. So, I deactivated facebook account and now using only messanger
Yeah sorry, I meant something more like "if you look at the goal of the original use case of Facebook" not "if you as a user simply use some subset of the website". The greed to maintain and grow the valuation of a publicly-traded company--and thereby to optimize the entire thing for explicitly only maximal revenue (and thereby maximal engagement)--has so universally destroyed the dream of social networking that we simply don't actually have a large social network anymore.
Facebook was originally gated to just university students from specific universities, then they started to open it to everyone. It was basically a university bulletin board.
Yeah: I am actually old enough to have been waiting for Facebook to be supported at my University ;P. I remember it already spawning with the same functionality of, say, Friendster, and so it already was based on the "social network" concept of people having "walls" they posted to and the concept of "friends" that you were thereby keeping up with... which frankly isn't anything like a "bulletin board" (virtual or physical) and was much more like people posting stuff on the door of their dorm. It could be that it was a bit different, though, in the first handful of institutions before it got around to supporting mine.
We don’t just need human moderation, we need due process. These companies control too much of our digital lives and make so much money from us, but have zero regard for us as soon as it’s inconvenient to investigate a case, because that would cut into their insane margins. I was permabanned for Twitter recently merely for getting the attention of some large influencers who disagreed with me (no actual rule was broken, but I received enough reports that my account was nuked).
I appealed multiple times but Twitter’s appeal process is a sham for us little people. I doubt any humans ever looked at my account.
Emphasis 'we', not 'my'. These guys are already making shadow profiles out of info given by friends, corporations, etc. Not participating is also causing red flags in certain circles. Withstanding peer pressure is one thing, having your identity made up or flagged out of your control is another.
This is a slippery slope that should be tackled before it gets to that. The only people not affected indirectly, are the people who will die without children or younger cohorts as friends.
> These guys are already making shadow profiles out of info given by friends, corporations, etc.
True, it's a shady practice indeed, but signing up and giving them even more information directly from the source is obviously far worse than the fraction of signals they can extract from your friends.
> Not participating is also causing red flags in certain circles.
There's no accounting for the peculiarities of social groups, you could say the same thing about refusing to smoke weed or drink alcohol, it doesn't mean those vices are vital, and social media is no different.
I don't think you fully grasp what the slope is sliding to. There are enough token anecdotes of companies doing background checks on social media, and will actively flag individuals for having zero presence. We also have social credit score horror stories.
Your answer doesn't work anymore when lack of participation is considered wrong. We should be blocking that instead of assuming things will just work out forever as long as individuals guard their identity. Again, this goes beyond just standing up against peer pressure.
> There are enough token anecdotes of companies doing background checks on social media, and will actively flag individuals for having zero presence.
I'm sure it happens, but I don't believe that to be a real issue since using the presence of a social media account as a filtering tool for hiring is obviously ridiculous, and as someone who has done a lot of hiring, it's completely absurd to imagine we'd ever turn away a good candidate because their name didn't hit on a social media search, especially because it's very common for people to use nicknames or false names on social media or to completely remove their account from search altogether.
I also don't see the peer pressure thing as an issue. Adults don't meaningfully peer pressure other adults to use social media, nobody cares, and kids will peer pressure for everything from video games to sex and drugs, but it's pretty obvious that being peer pressured to do drugs isn't a valid reason to use drugs.
I'm not sure why you circle back to peer pressure when we agree it isn't an issue. Are you reading past the comments?
>it's completely absurd to imagine we'd ever turn away a good candidate because their name didn't hit on a social media search,
Understand for a moment many of these people are not developers with well-established CVs. These are normal people working the bottom of the ladder where there are plenty of replacements, and the answer to being irreplaceable is effectively 'start becoming a prodigy, establish a network early or be lucky'. Often too late for them. Even that advice alone is insane for the yet-to-be-born given a virtually global mental health crisis.
Leaving things up to executives behaving in a sane manner has given us multiple global problems to deal with. I wouldn't count on their sanity to prevent another.
> I'm not sure why you circle back to peer pressure when we agree it isn't an issue.
You're the one bringing it up. You've mentioned peer pressure in all of your replies.
> Understand for a moment many of these people are not developers with well-established CVs
It doesn't matter the industry or the CV, the idea that the absence of a social media account factors into hiring decisions in any real way doesn't make sense.
I think there was a remote possibility of it being an issue 6 or 7 years ago. Since then the problems with Social Media. People preferring not to participate, has become such a trend that it is extremely unlikely to be a hiring issue these days.
However, in *some* corners of corporate HR, it's certainly been part of their scoring, at the peak of Facebook's popularity.
You haven't thought about peer pressure enough. It's got depths.
"You must do this thing" evolves into "you must support this thing" and then into "absence of support is equivalent to nonsupport (antisupport... whatever)".
1: reach their arms around and feel in the middle of the back
2: arch a bit back and fourth
3: realize they have a spine
4: say "i dont give a shit if you are in facebook or twatter or whatever, I am not. Deal with it."
I hate social media, and use it as little as possible, but can't figure out how to do what you are saying here in practice without total social isolation in the real world outside of social media.
For example, there are several sports I participate in (physically, in real life) but these are organized on either Instagram or Facebook. I have created accounts solely to access this information (date/time of events). Facebook and Instagram are constantly disabling and blocking my accounts, apparently because my low engagement (zero posts, only "lurking") triggers some sort of bot detector algorithm. I have no recourse, and can't contact anyone at Meta about this.
I've tried getting these communities to inform me outside of facebook/instagram, but it's too big of an ask. These mediums work for everyone else except me, and the people involved lack the tech savvy or interest in trying to find an alternative.
Unless you don't interact with anyone, that's a completely untenable position due to network effects. eg My family (spread around the world) uses WhatsApp for communicating with each other, which I'm not much of a fan of but it's pretty much impossible to get them over to another platform given that I don't live anywhere near them anymore to teach them and they have to use it anyway for communicating within their residential community etc.
Sure technically I'm not being coerced into using WhatsApp, but it isn't exactly reasonable to say that if I really cared I would just not talk to my family until they figure out how to use a platform I prefer.
WhatsApp isn't social media, it's a messaging app, but more critically it's based on phone numbers, so WhatsApp really has no control over access to your contacts.
> Unless you don't interact with anyone, that's a completely untenable position due to network effects
It doesn't have to be this way though. Between sms, email, telegram, signal, and discord I have communication channels to every person I actually care about, and it's trivial to bridge additional layers of communication if needed.
> it's pretty much impossible to get them over to another platform
I hear where you're coming from, but in my view this is an intentionally defeatist attitude. We're throwing up our hands and saying "social media owns us and there's nothing we can do, it's just too hard to install another app". In reality, if it's important, it's not that hard. There's no disputing that social media is convenient, but it isn't vital.
I am aware that WhatsApp isn't social media, it's just an example of an app I would ideally like to switch away from.
I'm not really sure how it's defeatist when trying even to only get myself off the app would require making things much harder for my relatively tech illiterate parents on the other side of the world with no tech literate relative to lean on to help them out. With Whatsapp they've used it for a few years and can easily get help from any young neighbor in case of issues.
It isn't like I'm not trying. For example, for communicating with some close fairly tech literate friends, we go through a relatively big effort to host and maintain our ideal of a self-hosted Matrix and Misskey node. But there we can manage it due to everyone in the group being able to at least describe the errors they run into.
I haven't used Facebook or Twitter in well over a decade.
Nothing is more freeing than not having to put up with entitled, whiny idiots who think they are the moral authority on pretty much everything in the world. Especially since most of them couldn't accurately point to country that isn't America on a map.
These companies control too much of your digital life. I do just fine without having a Facebook, twitter, instagram, gmail, etc…
The only company which truly has me by the balls is Apple. There isn’t much getting around it however because you are either going to get boned by Google or boned by Apple, and contrary to social media accounts I think that cell phones are essential.
It's similar on Reddit and, yes, even HN. The mere fact that you were "flagged" (by one of your peers) means that you are, to a significant degree, flagworthy and thus justifiably treated like a criminal.
Maybe these people doing the flagging are special people who have proven their worth. I dunno.
> Before I appealed they were simply going to not show the picture.
I wouldn't be surprised if Facebook didn't just "not show the picture" and they actually forwarded your info to police or some three letter agency. You're probably flagged already. Probably being watched more closely as a result of your "perversion" and clearing up the matter with Facebook may not matter to anyone else they shared that particular data point with.
I agree you should get the hell away from facebook, but this kind of thing won't stop there. Apple wants to scan your personal files and so does Microsoft (last I checked Windows 10 already records and sends to MS the filename of every image you open in their "Photos" app and how long you spent looking at it). When your OS is acting against you, or even your cameras there won't be anything you can really stop using.
I sometimes wonder what would happen (it won't) if some fed-up Congressperson drafted a bill that would allow folks to sue Facebook, etc. for libel when it accused you of being a ped0, criminal, or other malcontent. Perhaps they would be a bit less happy to accuse their users of vile things.
More in the way of a specific law. I don't think a full Sec 230 repeal is a good idea. I really, really want services to have to specify what you did wrong, and you should have some actual options to deal with these services. They have become utilities and need to have some accountability when they call you something vile.
> Except I'm terrified to imagine a world where Meta might have called police on me based on the output of a neural network image classification.
Don't worry, Google's already doing it. And destroying your digital life if you've put too much stuff in their care. With the characteristic Google customer service of "go fuck yourself".
"We" don't need to retreat back into smaller forums. Despite the huge number of false positives flagged by content review systems, they still impact less than 1% of active users. So everyone else will continue using it. Facebook is terrible and unethical in many ways but it's still the fastest, most convenient way to share pictures and updates with friends and family scattered across the world. I don't have enough hours in my day to pursue other options.
Perhaps, but so what? At that scale there will be huge numbers of both false positives and false negatives in any content moderation system. If you dig into any popular online classified sales site you can find some illegal items.
Criminal activity is quite a different thing than censoring legal content which possibly violates corporate terms of service. If you have evidence of an actual crime then you should report that to law enforcement instead of expecting a private for-profit company to handle the incident.
And absent further hard evidence, I am frankly skeptical of your claim. Most people aren't experts on the nuances of firearms sales laws in various jurisdictions, so a post that appears to be soliciting a crime might be entirely legal (or vice versa). I don't really use Reddit, but I've been on Facebook for years and have never seen a post for an illegal firearms sale. Do you at least have some screen snapshots?
Far less important of an example, but I was just put in "Facebook jail" for 24 hours for posting a picture of my son at the beach in his bathing suit with no shirt. Y'know, as one does at the beach. I can only assume it's because my son has long hair and the Convolutional Neural Network or whatever decided he was a girl and therefore I'm a pervert.
Sadly it was the "appeal" I submitted that got me blocked. Presumably by a "human", but who knows.
Before I appealed they were simply going to not show the picture. Appealing got me in "trouble." That might be even worse than the original misclassification. On top of that, if I was actually a "community standards" violator who posted potential child exploitation imagery, I'm not sure how a 24-hour ban on activity on Facebook is of any use, either? Except I'm terrified to imagine a world where Meta might have called police on me based on the output of a neural network image classification.
Others have said but I'll say it again: This kind of business doesn't scale ethically. You can't have billions of people on a bulletin board. It doesn't work. Moderation is essential to modern communication. But you can't do moderation automatically or at scale and in a universal way.
Very dark patterns emerge the moment you go FAANG scale and toss algorithms tuning for advertising and "engagement" into the mix, and attempt to do so with the help of computers.
"Sad" as it is, we will need to "retreat" back into smaller forums and BBSs where communities self-police.
Facebook has infiltrated so many aspects of society. Want to interact with the parents from the local school your kids go to? You have to do that on Facebook. Event announcements? Keeping in touch with your distant aunt? Facebook.
If something like Facebook is really a universal utility, it will have to be put under public administration; like the post office. But clearly, that isn't going to happen and would have other problems.
I am going to have to find some other way to engage with old friends and family.
Frank Herbert had some intuition with his whole "Butlerian Jihad" thing.