The web as a whole never came up with an answer to the question "As someone in possession of valuable information, why would I want to allow that information to be indexed, which would allow someone to trivially copy it?"
"Information wants to be free" was fine for ~1990-2010, until Google et al. took advantage of it to build walled gardens they could profit from.
In retrospect, I feel like if the in-web-standards micromonetization efforts had been adopted, we'd be in a better place today, because there would have been better revenue channels than "Whoever controls ads."
And/or differentiate and regulate search as a privileged common carrier-style business class, prevented from reusing their web scraping for other products.
I think a big reason why ads became the dominant way of revenue generation on the web (as opposed to micromonetization) was systemic issues that just took time to solve.
a) early internet adopters were mainly young people who didn’t have a lot of purchasing power yet
b) a complete lack of trusted means for online payment
More recently things have started to trend away from this because these two issues are now solved. You can see this in social media like Discord or Telegram who have a freemium model, movie/tv streaming like netflix or disney plus, new entrants like kagi or the general proliferation of SaaS offerings which can now sustain a premium userbase where previously you would’ve opted for an ads based model (think of the n different todolist providers etc.)
Ads have existed across mediums though, from print, to TV and now on the web. IMO they reflect a fundamental unwillingness of the general public to pay for information.
I think that for the majority, the issue isn't really the money, but the friction. Having to set up an account, or pull out a credit card, is an enormous friction point. The path of least resistance is to tolerate the ads.
A proper micropayment system could greatly reduce or eliminate the friction of payments, but we still don't really have one.
It certainly feels like there are some regulatory loopholes/oversights being abused.
Maybe just make it illegal (or actually just enforce existing laws) to link to websites that violate copyright laws (from pages with monetization) and force the market to sort things out?
It would be the end of an era for the internet in many different ways, but maybe the wild west needs to just end?
There are serious issues around that, though, especially if the restraint is based on copyright violators. Copyright law (at least in the US) is a hot mess and unfair all around.
And there's a clear free speech aspect. Why should I not be able to mention the URL of any other place on the net? If there's a drug dealer in my neighborhood, there's no law (nor should there be) saying I can't tell people where the house is. Why should it be different online?
"Information wants to be free" was fine for ~1990-2010, until Google et al. took advantage of it to build walled gardens they could profit from.
In retrospect, I feel like if the in-web-standards micromonetization efforts had been adopted, we'd be in a better place today, because there would have been better revenue channels than "Whoever controls ads."
And/or differentiate and regulate search as a privileged common carrier-style business class, prevented from reusing their web scraping for other products.