Don't get me wrong, from a user point of view it is. Nonetheless, it impacts the business, no-doubt. A movie theatre may want to embed adverts for other films down the side of its listings, it may have other reasons for wanting people to actually visit its site.
Google isn't doing harm from a user perspective, but it palpably is doing harm from a content provider's point of view by screen-scraping the content and repackaging it as its own.
I walk into the shop, I see some newspapers with headlines on them, I decide to buy the paper, or I don't.
If I don't I'm not 'stealing' anything.
If a publisher wants to show the headline story to prospective customers in the shop, it can -- google will display the headline, and if people are interested they'll click, just like they'll buy the newspaper in the shop if they want to read below the fold.
If the person isn't interested they won't click / won't buy the paper.
If the publisher doesn't want people in the shop seeing the headline, they put it inside their publiccation. Online they simply mark it as unavailable to google (robots.txt or whatever).
Now if the argument is that search engines have no right to read your site and display the headlines without explicit invitation, I'd argue that operating a web server is the invitation to all, and robots.txt is the bouncer saying "you aren't allowed in". Google and other search engines obey robots.txt / the bouncer.
> I walk into the shop, I see some newspapers with headlines on them, I decide to buy the paper, or I don't.
If I don't I'm not 'stealing' anything.
Are you doing this on an industrial scale and building a large ad-supported business from it. More importantly, are you standing outside the shop saying 'don;t go in there, old chum - I can tell you what's going on'.
Either a simple robots.txt (a sign telling school kids they aren't allowed), or a username/password (members only), or even close the shop completely.
The newspapers have headlines and the start of stories on display in shops, trying to attract passers by to their wears. You can even read the first few sentences of the cabinet meltdown in the telegraph, far more than you can on google, all without paying the newspaper owner, or indeed the shopkeeper, a penny.
The headlines are put out there for everyone to see to entice people to come in and spend money. If the shop or newspaper proprietor doesn't want people reading the headlines without buying, they can not put them on display (robots.txt), or cover them up, like on these (ex) publications
And that's fine. I think it is perfectly reasonable to have good display a headline and first sentence snippet. What isn't reasonsble is if Google were to mine the story for info and display the whole lot on their site, rather than on the newspaper's site.
Yes, but at the end of the day it should be all about the user, right?
For example when considering the cost of water pollution we don't try to balance the interests of the owners of paint factories with the interests of people who drink water. Instead we consider if the cost of compliance with new antipollution laws will be worse for people who use paint then the cost to them of drinking polluted water.
It is all about the user in the long term. Google dominating and closing the internet is very problematic. In the long term, we could have users that do not go out of Google and just accept whatever Google give them as THE answer.
Maintaining a healthy ecosystem and competition is good in the long term.
There are many things that are forbidden even though the consumer might like them in the short term (ex: selling lower than the cost for retailers to kill all attempt of competition).
Google isn't doing harm from a user perspective, but it palpably is doing harm from a content provider's point of view by screen-scraping the content and repackaging it as its own.