I think what folks are missing is that a lot of these "zero-click" searches happen as a result of Google scraping your website, and displaying the results as a "featured snippet."
Yes, they link to you below the featured snippet.
No, more people don't click, because they've taken the answer from your website and displayed it right in their search results.
For example: If I'm searching for "best nail for cedar wood" Google gives me the answer: STAINLESS STEEL - and I never had to click through to the website that gave the answer: https://bit.ly/2MdovdP
• Yes, this is good for users (it would also be good for users if Netflix gave away movies free)
• Overall, the publishers who "rank" for this query receive fewer clicks
• Google earns more ad revenue as users stick around on Google longer
Ironically, Google has a policy against scraping their results, but their whole business model is predicated off scraping other sites and making money off the content - in many cases never sending traffic (or significantly reduced traffic) to the publisher of the content.
No, more people don't click, because they've taken the answer from your website and displayed it right in their search results.
It's for this reason that's I've stopped embedding micro data in the HTML I write.
Micro data only serves Google. Not my clients. Not my sites. Just Google.
Every month or so I get an e-mail from a Google bot warning me that my site's micro data is incomplete. Tough. If Google wants to use my content, then Google can pay me.
If Google wants to go back to being a search engine instead of a content thief and aggregator, then I'm on board.
I just got one of those emails for the first time about my personal site that's basically my resume. Apparently my text is small on mobile (it's not...) and some other crap
I don't get why google thinks it's acceptable to critique my site without prompting. It honestly just feels rude. They want me to do a whole bunch of micro-optimizations on a site that already works fine because it doesn't fit their standard of "high quality". I think I've gotten exactly 0 clicks from Google search results ever and I don't really ever want any.
If it were possible to get a human's attention at Google I'd start sending my own criticism their way but of course it doesn't work like that...
I was curious what it was complaining about, since https://henryfjordan.com looks great to me. I tried to run it through Google's "Mobile Friendly Test" but fetching failed [1] because your robots.txt has:
User-agent: *
Disallow: /
This would explain why you've gotten zero clicks from Google (or I would guess anyone else's) search results!
On the other hand, it's surprising that you would get a notification if you had crawling disabled. Did you set this robots.txt up recently?
Google seems to see robots.txt as "more what you call guidelines, than actual rules". Sites that block googlebot or all bots with robots.txt still turn up in google searches, just without a description, and are obviously still indexed.
robots.txt is a tool to control crawling, not to specify how you would like your site to be displayed (or not) in search results. If you don't want search engines to include your site, set:
The blog post you link has a bunch of alternatives, but I agree they're not great. If there are a lot of webmasters who want to be able to noindex through robots.txt then making the case for adding noindex to the standard would be a good next step.
I sent you an email, and I'm posting it here but without identifying info:
---
Hi Jeff,
Thank you for your comment, I'm replying via email to send some info I'd rather not share on HN, but will post the same redacted in HN. I used to (back when starting my web-dev career) run a one man show development team of a web agency and all our development/pre-prod sites (that had to be unauthed) had robots.txt to disallow all bots, but they still popped up in Google. Searching some of the old domains in google I found an example here: http://***.***/***, and attached is an example of it showing up in a SERP and a what the robots.txt looks like (and I'm pretty sure that the robots.txt has looked like that since that page was created).
In this case it is just one page that nobody will care about, and since I'm not working on projects that are open but "robots.txt hidden" anymore I don't know if it is as bad as it used to be, but I regularly see pages with the "No information is available for this page" whose domains have robots.txt's that disallow all bots but still show up in Google.
Thanks for sending the screenshot! That site shows up with "no information is available for this page", which means that while robots.txt has disallowed bots from crawling it the page is still linked from other pages that do allow crawling.
The robots.txt protocol gives instructions to crawlers about how they should interact with the site. If you instead want to give instructions to indexes, use the noindex meta tag.
You're right, I was wrong about how to expect a "Disallow: /" to work. But isn't it sorta odd to have a protocol to control crawling (which is usually done to index) but (almost) require a compliant indexer to crawl all pages to comply with the indexing rules?
In this example the robots.txt has clearly told all bots to not crawl this site, but the only way to read the meta tag (or equivalent header) is to crawl the site. So I assume that in this case google either assumes that it is fine to crawl URL's that it has found elsewhere while ignoring the robots.txt or it assumes that pages disallowed by robots.txt are "open for indexing/linking", which would mean that any page both disallowed by robots.txt and which has a noindex meta tag would still show up, right?
What is the intended behavior if a page is disallowed by robots.txt and still linked by another indexed page? Will it get crawled or just assumed to be okay for indexing/linking? Is there any way to tell Google not to index/link and not to crawl?
If you have a calendar where every month links to the previous and next months, a crawler can get stuck and hammer the server. That's the kind of thing robots.txt is for.
>"more what you call guidelines, than actual rules"
they can index without scraping. It is enough that other websites have links to you site. So the google bot follows the rules in robots.txt to the letter. "no-index" is the way to stay away from google.
They can't read my no-index if they obey my robots.txt. Do they break the robots.txt to be able to read my no-index or do they assume my "Disallow: /" means I'm fine with them indexing/linking?
Without the noindex part of robots.txt (which google decided to ignore not so long ago) this is not solvable.
Oh, I just added that yesterday as a response to the email. Before that I was actually running Google Analytics but since I get basically 0 clicks it wasn't really useful.
I have a feeling the PDF viewer triggered it, cause on Mobile it defaults to showing the whole page which results in tiny text but that's easily fixed by the user so I prefer to leave it like that.
Yeah it's amazing how rapidly and rabidly they show up when the complaint is on one of their paid features like a Google cloud (GCE) post for them or a competitor, but nada on the other products. Well no it's actually not surprising.
> If Google wants to go back to being a search engine
While I understand the problems with Google scraping content, as a user these snippets help me find what I'm searching for faster. If that's all you're optimizing for, Google is fantastic. There are certainly good arguments to be made for other models, but for search, stealing content helps. I'm not advocating stealing content, I'm just saying that it produces more useful results.
How do you know that the content Google features is the best there is? If we stop clicking on sites and just rely on Google to provide us the content we'll go down a very slippery slope.
I don't really see how this problem is any different to 'how do we know the #1 search result is the best content there is?', if it provides you the information you want, then great, otherwise you load #2.
Google lends the weight of its authority to the answers it presents. It's one thing if Infowars says that Obama is planning a coup against Donald Trump, it's another if Google says so.
The first three result lead you to fake android blog telling you how you can easily root every chinese android device and specifically the M89 tablet...
The real authoritative result (xda-developers) only appears in the fourth position, under sight. It will tell you if you follow the instruction given in the fake blog post from the 2 or 3 first results, you will brick your tablet.
In a similar way the word "cbd" (for cannabidiol) has been hijacked by dubious commercial compagnies through fake blog posts filling pages after pages of google results telling you how great cbd is for the treatment of every disease on earth... But there is no trace of an actual study in these results. You will have to go with the less popular word "cannabidiol" to start to see some serious articles about it.
Google results can be hijacked and Google do little about it. May be because the ads shown in these fake blog posts are from google ads network ? I don't know...
But google result have clearly deteriorated these last years and the authoritative figure of the companie is not anymore what it was in the past.
I know that sort of thing happens sometimes (Google presenting a spurious statement as a categorical answer) but those are bugs. As long as they are very rare, and fixed quickly when they occur, I don’t see them causing much harm.
OK, some people believe anything they read (especially if it confirms their existing biases), but that problem has always existed. I think Google’s occasional snippet fuck-ups are a drop in the ocean compared to the spread of false information through social networks.
There's the modern news-cycle axis, where Google can and should devote full-time engineers.
But the long tail is important too. It's fixed now (yay) but for years you could search for "calories in corn" and Google would confidently present an answer 5x the true value, scraped from a site with profoundly wrong information. As Google moves to present more direct answers and fewer links, this risk increases.
It looks like they have backed off on the direct answers somewhat which is good news.
Very few new blogs and content websites are being set up.
All content is moving into apps and walled gardens. Part of the reason for that is that running a well researched blog will never pay for your time, so becomes a hobby thing, and most people are fine to use Facebook for that.
> Micro data only serves Google. Not my clients. Not my sites. Just Google.
Well it also serves Google's users, to be clear. Though I should also be clear that I don't think that justifies it, since I think it's bad for the ecosystem in more subtle ways than are expressed in immediate user satisfaction.
That depends on how you define "users". If you define a website creator also as a Google user (by virtue of wanting to be found through Google), then Google is serving part of its users to the detriment of their other users.
And if you view Google instead as a connection broker, e.g. a middle-man between publisher and consumer, then Google is destroying their own business by snubbing publishers. Assuming that Google is still making rational, intelligent decisions, it follows that Google no longer sees itself like that.
Did Google ever see itself as prioritizing publishers and consumers equally? I think that’s a false premise and the parent is right; Google’s priority has always been consumer first.
Google (and virtually every other search engine) has always included content with links, what's different now (but not unique to Google, though they are perhaps the most advanced at it) is that now it algorithmically synthesizes content instead of merely aggregating it.
On top of all that, Google's snippets aren't curated and therefore, aren't always correct. They can be (and almost certainly are) gamed. Users that don't click through open themselves up to carrying on being misinformed.
I've found them to be incorrect so often on things when I would click through to the actual page or find a better link. I don't trust just the blurb for any answers any more.
I don't trust just the blurb for any answers any more.
I don't, either.
A site I used to own had a discussion forum on it. It contained a message along the lines of "Real Estate Agent X is a great guy. Real Estate Agent Y is a complete sleazebag."
The blurb that Google displayed for it was "Real Estate Agent X... is a sleazebag." And that was the first result for anyone who searched for that agent's name.
As you can imagine, I received many angry e-mails, phone calls, and legal threats. No, you can't explain to angry people that it's "just" an algorithm that told the world that they're a sleazebag.
I ended up editing the post so that Google would display a different version after its next scrape.
I think there's more to this... Google use lots of fancy Natural Language Processing stuff to extract that data, and unless the wording was very tortuous, I doubt it could make such a big mistake by chance.
They can get it painfully wrong last time. I came down with something like optic neuritis a few years ago. It's often one of the first signs of MS in many folk. When I googled something like "MS life expectancy", the blurb said something like "3-7 years" -- with subtext indicating it's 3-7 years LESS than average rather than "you're kicking it in 3 years".
I think they’re believable because google started by providing things that weren’t wrong. If you search for a time zone google shows it in your local time, if you search for currency conversion google does that. All those things that it’s done for ages, which were things that were also typically correct.
Then the snippets show up, and they are presented in a similarly trust worthy fashion. But the snippets are really just the really just the result of which ever site has the best SEO, and that’s often a really worthless metric these days. The time zone and currency stuff is easy, because it’s math, but opinions aren’t. The thing is though that even if google didn’t have the snippets, those sites that gets snippets would still be the top results that we clicked, and we’d still get the wrong information. That would probably be better, because it might be easier to spot obvious bad sources, but I still think there is just a fundamental flaw in how SEO professionals have learned to game the google bot to bring the world useless information.
I mean, part of it is certainly on google. No one in their right mind wants to comply with Google’s ranking terms, unless you make money from google searches. Which means a lot of useful personal blogs have dropped off the face of the internet, unless you’re really lucky to see them linked on a place like HN.
I wish libraries would band together and make a privacy focused and curated search engine, because librarians are actually kind of good at finding you the correct information.
It sucks. Sometimes the bold text is the exact opposite of the answer to the query I search for. It’s very misleading unless you click through and read the full context.
This is especially true where the answer is time-bound, which happens a lot in technical topics. Many times the snippet is for an earlier version of the language (but still with a high PageRank), or the Operating System (especially Android settings), and the most annoying at all: an ancient answer in an undated blog post.
Google is good at dating undated content. They keep track of the first time they've ever seen a bit of text, and assume it was composed then, even if it later gets copied to other sites.
The point is that Google frequently adds another level of incorrectness, that may not be identifiable without checking the source. This is pretty common on Wikipedia, and when people link to things in discussion forums, as well.
And anything Google does, is done at vast scale, which makes me, at least, think it might be substantially affecting society.
But that's the responsibility of that website. Of course it's bad if Google lists a site with wrong information as the first hit, but I think it's worse when Google blindly copies that false info and lists it as their own zero-click result. By doing that, Google itself takes responsibility for the information.
Although sometimes the site is actually correct and Google still gets it wrong by copying the info incorrectly or losing some context or qualifiers.
I loved zero-click results back when DucfDuckGo first introduced them, but I'm less enthusiastic about Google's implementation of them.
It's important to note that this is strategically incredibly important for Google because this forms the backbone of their voice AI. The better at answering questions directly, the better their voice AI becomes and that leads to a lot of future products.
AdWords is and always has been the goose that lays the golden eggs, none of Google's other initiatives have ever rivaled that revenue. That's why they put so much effort into bolstering and optimizing their search results pages.
Another reason is the use of add-ons such as: "Google search link fix - Prevents Google and Yandex search pages from modifying search result links when you click them."
I have stopped using Google a few years ago, but just in case I keep this (or similar) add-ons of my Firefox.
I have no idea of the popularity of such addons, but they would also impact the tracking that Google does.
It's been this way for ages, although for chrome (iirc) this is managed via hyperlink auditing [1] which allows google to track what you're clicking even though the link appears 'clean'.
The click through google redirect also allows them to track things like relevancy of the content and time on site (if you return to google SERP by clicking the back button), in-case the target site isn't using google analytics (unfortunately most sites do).
Any search engine is going to want to know what people click on so they can make their product better. For example, I just searched for [test] on DuckDuckGo and when clicking on the first result I see DDG sending a ping back:
https://improving.duckduckgo.com/t/lc?...
which contains which URL I clicked.
(Disclosure: I work for Google, speaking only for myself)
Startpage is an anonymizing proxy for Google Search, not a full search engine. Crucially, it doesn't determine how to rank results. If they decided to try to compete with Google, Bing, Yandex, DDG etc directly by bringing ranking in-house they would have a very hard time serving good results without being able to track which of their links were popular among users.
How safe are all these plugins we install to escape tracking? Are we trying to escape big tech tracking only to hand our information over to extension developers? Looking at network traffic often shows a ton of extensions sending data to some aws server almost perpetually.
Asking because I'm not sure of the answer to this question and lately I've become even warier so I decided to uninstall everything except things I absolutely must have like colorzilla, grammarly and full-page screen capture. For adblocking I use brave and never ever touch firefox, opera or chrome.
There's an extension that appends a share=1 parameter to all quora links to prevent them from forcing you to sign in in order to view a post. I like it but I'm trying to minimize my extensions footprint and I'd rather write my own script to perform the same script.
The question is, how do you get to be sure that an extension is safe?
> Google has a policy against scraping their results, but their whole business model is predicated off scraping other sites and making money off the content
Yea a couple days ago I was checking the Places API, which they’ve built off user-generated content and scraping Yelp and others. They charge $17 / 1000 calls for certain items and don’t you dare cache anything for too long.
Great way to build a business: get data for free, wall it off and put a hefty price tag on it, then put your best lawyers around the moat for good measure!
I downloaded all the places data for the world while it was still free. In my jurisdiction, the data is considered owned by the place owners rather than Google, so I doubt they'll come after me.
I disagree. There is an implicit contract between website publishers and search engines that it’s ok to do this. The website can set nosnippet in robots if they want to not have the snippet in search results.
You put a resource on an open network and don't use any of the standard, recognized methods to indicate don't index, don't share, (nor lock it away with auth).
It's like if you put a sculpture in front yard and get upset when someone points it out in their neighborhood tour company, even worse cause yard ornaments don't have standard accepted methods of saying "don't use".
You put a resource on an open network and don't use any of the standard, recognized methods to indicate don't index, don't share, (nor lock it away with auth).
This is the kind of argument people used to use as they flagrantly violated your copyright by cloning your article on their own site. "You put it on the Internet, so it's free for everyone to copy."
The law says no such thing, at least not in any jurisdiction that I'm familiar with. Contrary to popular belief in some quarters, normal laws do still apply on the Internet.
If you infringe copyright, it's still infringement even if what you copied was freely available on someone else's site.
And if you state something that is misleading and harmful, it might still be defamation, even if what you stated was just an automatically generated snippet that takes a small part of someone else's site and shows it out of context.
Nah. Take it easy here, there is a long way between indexing and showing the most relevant hit and outright lifting big parts out of the site and use them on their own property:
It is more like if the guide that used to send visitors to your property has set up their own boot on the best spot on the sidewalk next to you and are raking in money because of the useless (often, in the last few years) ads they have plastered all over it.
Even if it is an educational non-profit resource you don't want that as some of the details get lost when visitors only reads the guides summary instead of taking a closer look for themselves.
And according to people on this thread they will also complain and/or come with suggestions about how you can make it even more useful to them.
I think of it more as if you put a banner with content somewhere in the public, and I take a photo of it, what can I later do with that photo?
And for that, it's a question of copyright. It turns out, in the US, if something is publicly available it does not make the copyright a part of the public domain. Thus the original author still retains copyright unless explicitly stated otherwise.
There is an exception to this though, which is called fair use. And for that, I'd recommend reading this: https://amp.theatlantic.com/amp/article/411058/ Book snippets by Google searched were deemed fair use.
So the question remains, would website snippet similarly count as fair use? What will the federal courts rule be? And when it comes to fair use, that's the only way to know if it is or not.
It's worth pointing out in this context that the US legal concept of fair use is not universal. In fact, unusually for US IP laws, it's actually much more permissive than most other places. The more usual practice is to enumerate specific situations where copying without the copyright holder's consent is still allowed, instead of defining general tests, which is how fair use works. This has been a controversial point, because it's not clear that the US scheme is sufficient to meet its obligations under international treaties.
In answer to your final question, I'm not sure whether this use of snippets in search engine results has been tested in any US courts yet, but the issue of search engines showing enough content from the sites they link to that users never actually go through to the original site is sufficiently controversial that the EU's recently passed copyright directive includes specific provisions aimed at exactly that sort of situation.
> It is more like if the guide that used to send visitors to your property
Here is where your argument falls apart. The web is a public space - it's not your property or your front yard. It's more akin to going to the town square wearing a fancy hat and getting upset if people look at you and your weirdly shaped headwear.
The web is a public space - it's not your property or your front yard.
You're wrong here. Just because it's a public space does not mean nobody owns the property. As a simple example, a shopping street is usually a public place. That does not mean that all window displays, doorways and adjacent buildings are automatically a free-for-all.
In fact, only "the tubes" of the web are a public space. The rest is owned property, even if there are no visible fences.
Laws everywhere are pretty much saying your take is wrong. There is no such thing as an implicit contract, and your take on it is plain victim blaming.
It is very surprising to read this on a board where many people write code: if a dev found unlicensed code, they would certainly not think it is public domain.
It's a devil's bargain. If you opt-out of snippets, it simply means somebody else claims the top spot, and you are left with even less traffic (by a significant amount)
> I disagree. There is an implicit contract between website publishers and search engines that it’s ok to do this. The website can set nosnippet in robots if they want to not have the snippet in search results.
Who made this contract? I never signed one. If I came to your place of business and copied your content and provided it somewhere else, I would be infringing your copyright. Do I have to put up signs specifying that at my place of business? Why is this any different? My web content is not the property of someone else and by publishing my information that is in no way an implicit grant of the right to reproduce it.
I wonder how true that assumption really is any more. The quality of traffic Google drives to sites I operate is very low compared to all other major sources, with much less engagement by any metric you like, notably including conversions. The only reliable exception is when we're running marketing campaigns in other places, which often result in spikes in both direct visitors landing on our homepage and search engine visitors arriving at our general landing pages.
There is this conventional wisdom that SEO, and in particular playing by Google's rules to rank highly in its results pages, is the only way you can run a viable commercial site these days. Our experience has been exactly the opposite: our SEO is actually quite effective, in that we do rank very highly for many relevant search terms, but it makes a relatively small contribution to anything that matters. And really, when I write "SEO" here, I'm only talking about general good practices like being fast, having a good information architecture and working well on different devices. We don't change the structure of our pages just because Google's latest blog post says X or Y is now considered a "best practice" or anything like that.
Of course I have no way to know how representative our experience is. YMMV.
Yes you can. There are other ways to market yourself and your website. For instance, the author of “Fearless Negotiation” has appeared in four or five podcasts I follow. The well known pundits in the Apple ecosystem grew an audience organically through word of mouth.
Hoping to stand out on Google results as a business plan is recipe for failure. You are one algorithm change from going out of business.
Glad the first ranked response was this. It's what I came here to say. These days you simply don't need to click as often to get what you need out of a search, and Google's business model doesn't rely on click through to web sites, but for display & click through of ads.
Searching for "best car engine oil" has certain brands displayed straight on the featured snippet. Who cares about the click if Google found your customer for you and got your message through for free?
In the end, Google should care. If a search for "best car engine oil" got your product featured, that means you won a sale. But assuming the sale happens completely offline, Google lost its opportunity to inform you of the search, and of the succesful search->sale conversion.
That means your marketing department can no longer justify investing money in Google SEO, which means less optimization towards Google's crawler, which means less reliable search results, which means less Google searches in the long run.
Increased profits from unknown sources VS decreased profits from known sources. The gained marketing intelligence may come at the cost of the bottom line.
I 0 click search more than I click because but not limited to:
1 ) to get the correct spelling of a word that spell check cant find a suggestion
2 ) avoid going to a site that I might potentially get malware (example searching music lyrics)
3 ) avoid having to deal with slow loading and bloated pages
"• Google earns more ad revenue as users stick around on Google longer"
This one is actually reverse. Google search doesn't net google any money if people don't actually click the link, since ad revenue for google search is Per Click, not per view (per mille).
The incentives for them are actually reversed - increasing the amount of clicks into external websites, specifically advertised links, increases their revenue. (which is why there are so many advertised links on a search page)
I do a fair amount of grammar and spelling searches. Google often displays tips and examples. And typing "sp500" displays a stock chart right in Google itself. Google has a lot of "instant snippets" like that. Quite convenient. However, near-monopolies do make me nervous about supporting them.
Great point, this was my first thought also. Google has been doing a slow creep of this type of content for the past through years through the featured snippet you mentioned, and other knowledge panel material. They now serve sports, weather, math, translation, flights, etc.
Speaking of scraping, does anyone know where one can get a hold of full text news articles/press releases for nlp research? Most APIs that I have found only offer partial texts.
I know that Aylien has an API for this but it's out of my price range.
If I recall correctly content(news especially) publishers and some Europeans were very angry about that. I think the consensus was that these businesses don't understand the internet.
I don’t see any way to actually achieve this at scale let alone any reason to add an opening for more pointless lawsuits. Let’s say they’re liable and you choose to act on incorrect information recieved for free. Do you really try to take them to court and on what grounds?
Yes, Google and similar companies should be 100% responsible for anything published on their platforms. No more “safe harbour”. They have chosen to take positions in many issues, that makes them more like newspapers than phone companies.
Positions like what? And no, banning radicals from their platform for violating their terms of service is not a position.
Even if they were responsible, it's still legal to lie. You don't see pseudoscience websites being taken down because they are objectively false either.
It’s OK for the NYT to attempt to “prevent another Trump situation”. They have an editor and that person is legally responsible for what they publish. They don’t even pretend to be non-partisan. But Google takes a position then hides behind “common carrier” status. It’s not reasonable that they can pick and choose. Either they’re the phone company or they’re a publisher. It’s their right to be either of course, but they must choose.
> It’s not reasonable that they can pick and choose. Either they’re the phone company or they’re a publisher. It’s their right to be either of course, but they must choose.
This is 100% wrong, the opposite is true. The law explicitly protects website operators from being liable for content posted by 3rd parties while simultaneously granting them the explicit freedom to curate content that they deem objectionable.
No content on Google is posted there by 3rd parties. Google does select what is displayed and, in the case of snippets, they go out of their traditional way to promote that content.
The use of the word "post" is my own colloquially imprecise language, the law actually states
> No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.
So content indexed by google absolutely falls under the definition of "provided by another information content provider"
Providing links is indeed within this definition. However, cards go beyond that: by selecting one out of the many results promoting it and possibly alterating its meaning by selecting which parts, and how it is displayed goes far beyond merely displaying content provided by others.
Of course, there is plenty of room for google attorneys to wiggle, but in the end the objective for them is to 1) give credibility to a source and 2) to get the benefits of being the providers of information.
Common carrier and safe harbor are 100% separate and distinct concepts. The same way that a forum could have a theme ("political party X posts only") and still be allowed to remove illegal content is Safe Harbor (both curation at their discretion and no responsibility for illegal posts) - and I don't see how one could be against that - and Google is nowhere near that, whatever "positions" you envision them to have taken.
Google and other tech never claimed to be common carriers, and even internet service providers have been cleared of that status - barely anyone is legally required to transmit without discretion (it's pretty much just phone companies). So why make it about Google and Twitter, and start with ISPs?
I don’t mean literally alphabetically but according to some objective measures. In the old days it was by incoming links (PageRank). But now it is opaque and many people are finding that it orders by whatever is best for Google, not for the user.
There is not nearly enough room on the front page for everyone that wants to be there, google has to make subjective decisions about what shows up there, it's impossible to do it any other way.
I don't think that's a bad idea, but the vast majority of google users would not desire this behavior, especially the way google is used today where users try specific terms to relocate content they have looked up before.
Yet there is no spam in the Yellow Pages. It’s very unlikely that if you call Aaron he’ll clone your credit card or install hidden cameras in your house. Also, it’s very likely that he actually is a locksmith, has the accreditation he claims to, is a legitimate business registered at Companies House, fully insured, all the things you expect of a normal business.
Hold on. Google doesn't earn even a penny when you visit their site, find your answer on the search results page, and then leave. That user behavior COSTS Google money, it doesn't earn anything.
If they were trying to monetize you they'd show you an ad that links to your answer and take a profit on the click. Directly giving the user the answer they want is great for the user, but guarantees that Google won't earn any revenue.
So why does Google do it? Simple: because their competitors do. That's the free market for you. Google didn't start that feature, another competitor did; Microsoft made it their primary differentiating feature in fact (remember the "bing and decide" ads?). Google had to adopt the same behavior or lose their customers.
So no, don't blame Google, blame capitalism. This is precisely the kind of feature that you wouldn't get if Google was able to behave as a monopoly.
that is correct and should be considered copyright infringement... I am so tired of the double standard in the US of people VS corporations... Corporations are considered better people then real people.
But as Google sucks up the consumer surplus, it's going to be harder and harder to make money from internet businesses, and the final result a few years down the road will be toxic.
The internet isn't going to work too well if its solely reliant on hobbyists.
The funny thing is this used to happen. In the early days, you ask a simple question, you would get the answer in the search results, before they introduced feature snippets. The problem was, because no one was clicking on these useful sites, they were downgraded in the listings to sites that hid the useful info so you had to click on it.
This is the subtle truth that I've seen a few folks on Twitter talking about for the past year or so: That Google has slowly but steadily reduced both the outbound clicks to other websites, but also the portion of their revenue that's based on ads hosted by other websites, while bringing both the "results" and the ad placements in-house, where they no longer have to pay out a share to site owners.
Whereas Google was previously a way for sites to be discovered and for sites to generate revenue, it is increasingly becoming the sole source system where data is scraped and imported into Google, and Google keeps all of the revenue to itself.
The increasing sprawl of non-search widgets invading the search result page reminds me of the AOL years where "the web" was funneled through a narrow portal controlled by one entity.
Having to scroll down past ads, unrelated news, unrelated youtube videos, and ever more of these info boxes has pushed the actual content I'm looking for out to the second page. It's made it much easier to use ddg as default and use the !g flag only when absolutely necessary.
And the downfall of AOL was Google because Google had a better product. In order for google to fall, you need a product that's better (or more of what people want; you need to give them a reason to change their search engine).
I stick with Google but increasingly try to tweak my searches to hit forums. They're just that much less likely to be made-for-AdSense content by a copywriter paraphrasing other information from the web.
I often just add "whirlpool" which is a fairly reliable Australian forum that started covering telcos but these days will have things about cars, home maintenance, personal health, etc. Or I add "forum" and that can be enough to tilt the results.
I usually add site:reddit.com or site:news.ycombinator.com etc.
Actually google had a way to search discussion gruops, but they removed this feature as forums don't pay for their adds I suppose.
Same. I cant tell if my questions are just getting more specific and technical, but Google search results have been getting pretty useless in the past year or so
I love how google likes to completely ignore what I'm trying to search for. I wish I could think of an example because it happens to me often, but I can't so I'll make one up.
Imagine you're searching for tail lights for your car or something, but you don't know the size, so you search "Astra tail light size". This might bring up headlights. Wrong but no matter, you'd go on to google "Astra tail light size -headlight -head" or something.
What Google seems to have been doing to me recently is ignoring those negated terms, ignoring quotes, and just giving me the same results again and again. It's really getting annoying. Google seems to assume it knows what I'm looking for, and that my search query is just completely wrong and not what I want.
Note that the car stuff is just an example, I'd expect Google to not give you headlights the second time. It generally but not exclusively happens to me when searching things that are more technical. ESPECIALLY when it's a consumer level thing I'm trying to get info on, Google likes to assume it's giving you errors and you're trying to fix it. Which makes sense for most users, but god it's frustrating when every combination of advanced search parameters you try does nothing!
Google search needs a checkbox or something to turn off it's cleverness and just do an actual search.
Absolutely. I was trying to figure out something to do with timeouts in an SMT solver called Yices, so I had search strings about signals and alarm and Yices - of course. Google decided that this was a generalized programming question and displayed a lot of stuff about signals and alarm handling that didn't relate to Yices.
How likely a search time is "Yices", ffs? Feels like something that exotic ("statistically unlikely") probably is meant to be in the results by default.
I had no idea what Yices is. So I Googled it - the first link is SRI's Yices SMT solver. I tried "yices alarm" "yices signals" "yices timeout", and all of them showed only links related to Yices in the first result page (various manual pages, types, etc). So my attempt at reproducing your experience has failed.
The top Google hit for "yices alarm" is currently the exact Hacker News comment you've replied to. I wonder if Google adapted its search results based on that very comment? Maybe their algorithms shrewdly give more weight to fixing search results when the context mentions Google ("I googled for...", "Google didn't work when...", etc.) and the site is high profile (like HN). That would be very crafty.
Same here. It seems the websites that show up top are become more and more spammy and less relevant to my query. I keep seeing all sorts of one sentence hipster 2.0 sites that want me to believe they are a credible source of information.
... and that product will likely repeat the cycle again, on some schedule of another. Might be the 20+ years of Google, might be the few years that Medium was only modestly annoying, might instantly go to shit.
The problem is the business model breeds for this, and we end up replacing one abusive monopoly with another, until we can break that cycle.
For a time it seemed Free Software might ... free us ... from that, though as even that effort's biggest boosters (Eben Moglen, Bradley Kuhn, RMS) freely admit these days, we've been regressing of late, and at an increasing rate.
Anecdotally, i am noticing a steady decline in adsense RPMs for the exact same audience over past 2 years, and adsense is now most of the time not filling the ad slot.
Also, it has gotten very hard to rank in google for new sites. SEO blackhat tactics rule, and even local businesses use them. Google went from win win to win-get lost.
Sadly, now that webmasters need it the most, investments in alternatives to search and advertising have dried up. There is almost nothing except G.
I feel the same way about ranking new websites. In high school, friends and I made a website for which we did zero search engine optimization (nor were we even aware of search engine optimization). We still ranked in relevant queries and easily got an audience of about ten thousand uniques a month.
Today, I struggle to get sites I make to show up on Google at all. For my most recent website, even searching for phrases that are unique to my website doesn't cause it to rank. This is more frustrating to me, because I have a Google ad for my website, which drives all of my traffic - so I know Google knows about my website and what keywords are relevant to it.
I don't know how old you are, so it's hard to tell how long ago you're comparing. But how do you know if this isn't simply because there are a lot more content than it used to be ?
> Whereas Google was previously a way for sites to be discovered and for sites to generate revenue, it is increasingly becoming the sole source system where data is scraped and imported into Google, and Google keeps all of the revenue to itself.
I wondered yesterday: if you provide microdata, Google scrapes it, and you later decide to remove your sites from Google - is Google allowed to keep the microdata and continue to publish it?
If you once communicated the fact that 2+2 is 4 and Joe makes very good spaghetti you own the copyright to the text you published but neither the fact nor the opinion belongs to you in any meaningful way nor should it.
Not sure why you are downvoted. In the EU databases fall under copyright, which indeed leaves the question how google legally deals with this (technically database right isn't copyright, but in this context that's a technicality).
Also, to quote from the Wikipedia article [1]: An owner has the right to object to the copying of substantial parts of their database, even if data is extracted and reconstructed piecemeal
Because they didn't say "in the EU", and it not being copyright is not just a technicality. Copyright is about creative expression, and utilitarian collections of facts aren't.
They also didn't say "in the US". From context you can only assume "in some jurisdiction google cares about"
> Copyright is about creative expression
That's not true, or at least a very US-centric view. The Berne Convention, the international standard for copyright, reads:
"[...] shall include every production in the literary, scientific and artistic domain, whatever may be the mode or form of its expression, such as books, [...] works expressed by a process analogous to photography; works of applied art; illustrations, maps, plans, sketches and three-dimensional works relative to geography, topography, architecture or science."
also
"Collections of literary or artistic works such as encyclopaedias and anthologies which, by reason of the selection and arrangement of their contents, constitute intellectual creations shall be protected as such"
That's lots of things that are not exactly "creative expression" (even though exceptions for pure statements of fact do exist).
"by reason of the selection and arrangement of their contents"
If there was no selection or you make the original selection irrelevant, while also giving your own arrangement, then there's no violation of copyright.
No, but if Google imports a database then they are still affected by the compilation copyright. It's too obvious of a hack to "just" claim that "yeah, we imported that entire database, but then we cracked all the facts apart and they're all separate now and it's just as if we never imported the database". That's not how the law works.
Even more interestingly, we've still never yet resolved the question of why Google gets to lift your entire site's contents and re-serve them in arbitrary ways to their own profit in the first place. It's really just a thing that happens on the internet because it was happening on the internet before the lawyers got there. I've said before and still believe that if there was no such thing as a search engine and they were just invented today, they'd be annihilated in court as nothing but one big copyright violation.
I'd rather give up copyright than search engines. Anyone who wants to push too hard ought to consider whether an entire nation might make the same choice.
I am not sure there's a specific copyright applicable there. You can ask Google to remove your website's data from their index (primarily via robots.txt)... but of course, that also delists you from search. Essentially Google has left the impossible choice to either let them steal your data for free or accept not being findable in the primary search engine on the Internet.
Honestly, the second google started owning and promoting their own properties on search results, anti-trust regulators should have jumped on it.
I'm really not sure how Google can be protected by Section 230 and at the same time control and publish so much directly. Last time I read an article on the topic, google controls 23% of the top 100 sites.
I've heard a decent amount of wrong takes on Section 230. But this is the most bizarre, yet.
Neither the CDA, nor section 230 specifically, create the sort of publisher/platform dichotomy people seem to be hung up on.
And Section 230 does exactly the opposite of what people commonly think it does. It's actually right there, in the text:
No provider or user of an interactive computer service shall be held liable on account of (a) any action voluntarily taken in good faith to restrict access to or availability of material that the provider or user considers to be obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable [...]
That seems really easy to understand: you can delete nazi propaganda, porn, bad jokes, or just random user content from your platform without running the risk of thereby assuming liability for the rest.
But the inherently problem is that they need to be held liable for the rest. Because the rest is often criminal activity of which those platforms are making a profit. Perhaps removing Section 230 isn't enough, by your definition, because we need a law that explicitly holds platforms liable for profits generated from illegal activity.
Section 230 didn't prevent Google from being fined $500M for publishing ads for Canadian pharmacies selling drugs to US citizens. What kind of illegal activities are you talking about, exactly?
Google regularly distributes malicious websites and malware through ads, and refuses to delist reported malicious websites. (Google Ads is the primary distributor of Windows PC malware today, if my customer support experience is any indication.) And the problem is that Google has a perverse incentive: More bidders for ads means higher bids. Since malicious actors raise the prices, they benefit from bad actors selling ads.
And the problem is that even if someone finally comes in and shuts those actors down, Google kept all the profit from the malicious activity. In order to incentivize Google to police it's ad platform, we need to implement a requirement which seizes all revenue from malicious advertising, retroactively when a malicious account is flagged/reported.
If Google is losing revenue on allowing bad actors on their ad platform, they'll be incentivized to quickly respond to reports and remove them so that legitimate ads, which they make money on, can have those ad slots.
> More clicks on ads that lead to malware also leads to fewer clicks in the future, though.
I really doubt this works this way. There's 3 assumptions here, for your scenario to play out, people must pass the following funnels:
1- The person notices the malware
2- The person associates the connection between the ad and the malware.
3- After making the connection they install a non-dummy adblocker. Dummy adblockers like the one by Eyeo whitelist google's ads while actually harming the competition. It benefits them! Note: if I look up adblock on google, uBlock is only mentioned on page 2 of google and only because it's mentioned as a competitor to adblock on a zdnet article. The first whole page is dedicated to the Eyeo plug in.
I'd say very few people will get through that funnel. My experience is that when my family and friends actually seek out help with their computers, they have let it go for years until the computer is a slow mess of malware, self installed spyware in the form of browser add ons and other crazy stuff.
I've actually known a person who buys a computer every couple of years when it 'gets slow' simply to avoid maintenance. The few that I know from IRL relationships that do use an adblock, mostly use adblock by Eyeo, simply because of the domain and ranking on google.
But in the country I live in, the USA, landlords are liable for many things their tenants do.
A friend of mine is a landlord and he almost lost a house he owns because his tenant was cooking meth in it. I don't remember the exact details, but the liability was no joke.
P.S. I don't agree with this liability issue, I'm simply describing reality as it is.
When they get to know about that fact and do not take steps to end this behaviour, although it would be in their possibilities, they might be held liable, too.
So ocdtrekkies point stands, imho. Google is directly profiting from shady activities through their services and therefore has no incentive to control or stop that behaviour. That's a tricky thing.
My main takeaway from this was where the hell are they getting that data? A little digging… apparently if you install Avast antivirus they're tracking all of this and selling /providing to Jumpshot - wow.
> Baker said Jumpshot’s data comes from 100 million devices worldwide, whose users have downloaded free security software from partner Avast. The devices include smartphones, laptops and tablets.
errr... sure, if your antivirus is free. There are a lot of good paid antivirus that don't have these problems.
Of course, lately, paying for an antivirus hasn't had the same value - windows defender, the much better browser sandbox and the fact that you don't really download executables anymore have contributed to reducing the risk of going without an anti-virus.
I would expect the target groups "allows Avast bloatware on his computer" and "uses DuckDuck Go" are completely disjunct. Makes me wonder how valuable their data really is.
So a common trope in the startup world is the difference between a feature and a company, the idea being that lots of features are masquerading as companies. This inevitably leads to them being shocked--SHOCKED--when some bigger player adds that feature, eating their lunch.
I feel the same way about a lot of outbound sites on Google. There are a bunch of things I just don't want to go to another site for. Off the top of my head:
- Exchange rates. Although this one I find infuriating because Google doesn't know how to correctly round off exchange rates (as in, there's a standard). This manifests as, say, showing 2 decimal places for AUD/USD when they should be showing 4.
- Calculator
- Mortgage calculator
- Song lyrics
That sort of thing. If you go to any sites that provide these sorts of things they're typically "scummy". Lots of ads, lots of Javascript, lots of dark patterns to make you load more page views (eg a mortgage calculator that'll mysteriously take 3 steps/page loads to calculate).
I'm glad these are in search results, typically in significantly better versions. And I don't think anyone who runs a site built around a basic formula for interest calculations has any right to complain about it.
Of course this will be painted as "where does it end?" but not every surface is a slippery slope.
Just look at the likes of Yelp who complains about Google "stealing" their content. Well, Yelp is about one of the scummiest businesses out there. So I won't feel sorry for them, not now, not ever.
The one weird case here is AMP. Like I get Google's motivations here. Many companies develop terrible mobile sites that run badly or not at all and AMP IS much faster, generally speaking. Yet still it seems so heavyhanded with seemingly no opt out (on the consumer or publisher side). I don't really understand why Google wants to die on this particular hill.
I understand your stand, but what Google did with its featured snippets is essentially a form of bait and switch. It encouraged entrepreneurs and creators to create content that could answer specific queries with the promise that Google will drive traffic to their sites.
Now Google is using the same content and depriving them of the traffic.
User: Hey Gateway, what's the monthly payment on a 20-year mortgage with 3.9 APR?
Gateway: Here's your calculator, already prefilled with the data taken from your query.
Done.
The problem with all these little sites is, besides bloat, that the "Oh, neat, they can hook me up with lenders. Let me take a look." ends up with user getting malware and/or scammed. The problem with Google is that they can hardly be trusted at this point to do things like this in the interest of users. They want to be the frontend through which you access the Internet.
I used the word "Gateway" in my example as a placeholder; my imaginary perfect world recognizes that things like "currency conversion", "song lyrics" or "mortgage calculator" are data[0], which should be separate from the frontend used to access it. I dream of the Internet where things like these are API-driven and do not involve loading anything other than what's requested - neither ads, nor "value-adds", and definitely not all the rest of the webpage surrounding a mortgage calculator, which is bloat obstructing requested functionality.
--
[0] - yes, even mortgage calculator; it's a mathematical model, an algorithm, and code is data.
It's not even malware/scams - it's just "Person who wants a mortgage calculator" is "person who is interested in getting a mortgage", which is some seriously high-value data. There's a lot more profit in selling you on to mortgage lenders (with a lot of data, like your budget, pre-filled) than selling you to scammers.
And surprise surprise... That's what DoubleClick (= Google) does, they'll sell your cookie ID as "interested in mortgage" audience to advertisers.
While in the interest of "privacy" Google limits the amount of data the banks and others you interact with get from any competing solution, they will sell all of it in Ads Data Hub which is a Google Cloud product. So not so much in the interest of privacy but much more in the interest of Google selling more cloud products.
Are you arguing that everyone should be able to buy a house, that they can live in forever, with cash they have with little savings?
The problem with that is that if a poor person can afford a house with cash, then a slightly richer person would just go, "wow, I can buy a really big house! Or two!", and suddenly there is no more space.
Maybe they wouldn't be allowed to buy two, because in "a perfect world", a place to live is a necessity, not a profit vector. Keeping housing supply limited to boost house prices, to appeal to house-owning voters and wealthy landlords, to extract six figures of money from normal people over decades, is nothing like "perfect".
This sounds a lot more like inviting a trolling argument than any kind of genuine inquiry.
Here I say "landlord epxloitation, profiteering from necessities, houses left unlived in as investment vehicles for the international super rich, NIMBYism and entrenched residents voting against housing stock, AirBNB, broken zoning incentives" and then you say "FREE MARKET it's rich people's right to buy up everything and if you disagree you're dumb, free market is best market". And then we disagree to agree and both leave unhappy. Is that an approximately good summary?
We don't need to determine who gets which for this comment chain, we only need to question whether a 30 year mortgage should be the default way for a normal person to keep the rain off their bed, someone who isn't thinking of a beach condo in Malibu or a NYC penthouse, whether that is the best possible "perfect" world. I think it isn't.
And yet if you type in a mortgage-related query, the first few entries Google will show you are ads. Not the calculator you're looking for.
Even then, mortgages aren't something Google can realistically calculate because lenders don't structure loans that simply. There is a fixed rate period, and a variable period after that.
First-time searchers may get suckered in by a personal finance site that relies on ads just as Google itself does, but after that, they'll be using the calculators that are on lenders' own sites when it's time to comparison shop.
The ads in search results are a utterly, completely different character to the on-site garbage parent is talking about. They are simple text entries that are clearly marked.
The majority of UK home mortgages are 25 year with fixed rate for 2-4 years followed by variable rate(which if course no one ever goes on, you simply remortgage as you're about to go on the variable rate instead).
sometimes I try to mess with sites that ask for an email address for a mailing list by entering their own addresses: like "support@domain.com", "webmaster@domain.com", etc.
let them deal with their own spam, then they'll understand.
That's a great idea. I'll try that next time. My version of petty revenge is to sign up with one of my throwaway Gmail accounts, and then mark them as spam as soon as the first newsletter rolls in. Hopefully, there's others doing the same and they'll be at least partially blacklisted.
I think I may make the effort to learn how to make a Firefox extension to automate this. Scrape "about us", "contact", etc. page in background for first email address, then autofill it into the email spam form. Boom, done.
In some places this a growth hacking/marketing tactic (that works), in others it isn't the marketing team but someone up high demanding it because they've seen it. Am still a marketer, but I've been in the latter and trying to fight the good fight.
I'm not related in any way to that scene, but what would you tell to marketing when they tell you that that while the mailing list pupup is obnoxious and annoying, it results in X users enrolling and later on Y of them actually ends being customers?
There does seem to be a sizeable number of people who take any request on a webpage as a command, and comply with the email popup. Which would then generate leads.
But, this popup harassment is just going to turn away experienced users. Many of whom will be the educated, well-compensated demographic the site most wants to cultivate. And a subset of that group will have the ability to create or edit blockers, which will then be turned against the offending site. Thus making it more difficult for that company to reach the more desirable users.
That's what I would tell marketing. That wouldn't work for all companies. Some will be thrilled to get a boatload of inexperienced or naive users. But would be nice if the others would start to get worried about what they're losing with their bounce rate.
Are you asking for a hypothetical scenario where I was the web developer in question having to stand up to some marketing drone?
Take a closer look at the key phrase in my post: "makes no effort". If a web developer who's trying to pay the rent makes even a single comment to the marketing drones about how email signup forms might not be a good idea, and asks if they're sure they want to do it, then that web developer has my sympathy and not my hatred. But if they just cheerfully say "Yes sir!" when the marketing drones make that demand, then I hate them just as much as I hate the marketing drones, because at that point they are marketing drones.
I was asking because I have been thinking if there are any convincing arguments or incentives for avoiding such tactics, besides moral and integrity. I'm afraid that if's all depend on moral and integrity it is unlikely we are going to win this battle.
Morals and integrity are foreign concepts to marketers. I used to think that marketing had a place, if properly used. After working with marketers for a decade, I just consider them and their whole wretched field a lost cause. I have no sympathy, no empathy, no compassion for any of them. Nowadays I make every effort to use only paid services whenever possible. (If any Fastmail devs are reading... I frickin' love you guys.)
Your second part is wrong. According to the data most users don't dig deeper, they just stop at the answer Google already gave. MortgageSite doesn't even get the traffic in the first place.
I was using that example to establish that the sites (that are losing the clicks) are just as scummy as (if not more than) Google for "stealing" the clicks. That, in turn, helps explain why users might be reluctant to dig deeper, showing that general scummy practices are at least as much to blame as Google "stealing" their content.
Those sites or sites like them made the content initially that google now steals.
I remember a world without lyric sites. These sites provide value. The top links google shows may not because of page layout or bloated sizes but those are the links google presents there are lyric sites with no javascript. Meanwhile tons of mortgage related calculator pages exist on most bank sites and other non-spamming pages.
Clicking a google search result takes seconds to load the link while clicking a ddg link feels much faster. The additional tracking and redirections start to make things feel sluggish.
It's a chicken and egg problem. Website owners are forced to monetize more aggressively as the traffic from search slows down. And as they monetize more aggressively, the user experience suffers, causing them to tank in the SERPs, fueling even more desperation.
I agree that paywalls seem more common. But the early web had plenty of popups and scummy ads. Popup blockers were added to browsers quite a long time ago.
Content farms were huge before Google started kicking them out in 2011. Have they made a comeback?
Google's latest attempt to keep the crap under control (AMP) is very unpopular on Hacker News, but I guess it shows they are trying to do something?
> Content farms were huge before Google started kicking them out in 2011. Have they made a comeback?
Have they not? Content marketing seems as alive as ever, if not more.
> Google's latest attempt to keep the crap under control (AMP) is very unpopular on Hacker News, but I guess it shows they are trying to do something?
Because we - myself and many of those other HNers disliking it - believe that with AMP, making web more performant and less crappy is only an excuse, and one that doesn't stand up to scrutiny.
It's hard to tell since there is little in the news about them, but it seems like once, content farms were fast-growing businesses, and now they are bottom feeders? Or maybe they're just boring compared to all the other threats nowadays?
So you blame Google for paywalls and full screen ads on the pages... you demand they send you to in lieu of just hosting some content themselves? Seems like Google just can't win in that scenario.
FWIW: I agree that for a long time, the need to rank highly in Google's searches pushed content sites to be a better experience. And the equilibrium we've reached isn't as nice as it was 10 years ago. I just don't think "because Google" really captures the complexity here.
Google's whole value proposition is sending me to the best place on the web for a given query -- so yes, when they don't do that it's frustrating.
No, I don't blame them for hosting the content themselves for some queries. I never said that.
Just as Amazon holds some responsibility when I purchase a counterfeit item on their platform, it behooves both companies to have higher quality standards -- and both have the market strength to do it.
It's hard for me to shake the vague feeling that this feels like someone setting up on the sidewalk in front of your property and saying hey, you could have done X, Y, or Z, to stop me from being here. And you say, I did do X. And they say, oh, X was two years ago. Y was last year. Now you have to do Z. Until next year when it'll be AA. And weirdly, AA is the exact opposite of X, because what you thought would take care of things two years ago now penalizes you.
It's not a perfect analogy, because obviously people still hope for referrals from Google. But this idea that because something is published somewhere, it's okay, reminds me of Arthur Dent finding out that the notification about demolishing his house was in the third-subbasement of a local government building in a file cabinet marked "Beware of the Leopard." From rough memory; please forgive if I mis-remembered.
Do I get to bill google for all extra traffic I have to send by appending that to the body of my websites in order to prevent them from stealing my intellectual property?
Also going forward you have to add this to your site or I will steal anything that isn't nailed down.
I deliberately opted out of snippets for TLD List [0], a price comparison site I made for domain names.
I don't know if it's made any difference. Organic search traffic from them has slowly but steadily increased over the years.
Google now sometimes displays a snippet from my competitor's website for searches like "cheapest .io domain" [1]. The snippet seems pretty useless as it doesn't include any registrars' names/links (and my competitor's price info is quite outdated).
In these cases, since the snippet is the 1st thing users see in SERP, and doesn't provide enough info to fully answer the question, I'd wager that my competitor is ultimately receiving the majority of clicks from these snippets.
No I don't think so. Thanks for pointing that out.
IIRC, I eventually removed nosnippet because it caused google to not display microdata in SERP (see the "$25.99 to $99.80" in the above screenshot) that were desirable for my traffic. I instead replaced nosnippet with:
```
<meta name="robots" content="noarchive">
```
And this seemed to have the same effect as nosnippet, but with the added benefit of my microdata still being displayed in SERP.
I prefer Googles snippet to visiting a shitty site. This is one of the few instances that I fully support Google. If websites want to stay relevant, they need to be more inventive.
Yep. And what's worse is that sometimes the site is correct and/or up to date and the Google snippet attributed to that site isn't or is poorly chosen for the question.
Google "chancellor" from the UK and it'll suggest the question "who is the UK chancellor now" which expands to name the previous chancellor (an error Google attributes to gov.uk which would have been updated in July with the new appointment). Click on the link to Google the question and this time the snippet attributed to the same source will be up to date and give you the correct answer. But if I still need to load a new page to get the right answer I'd preferred to go direct to the source page without it serving me wrong answers wrongly attributed first.
And the other questions are also a rabbithole of rubbish to fall down (the question "who was the previous chancellor?" answered by "previous chancellors have opted for whisky..." is my favourite combination) or excerpt the wrong part of the page and then encourage you to search Google again to get the same unhelpful excerpt again rather than deign to look at the source period.
Getting this stuff right across millions of queries and unstructured data of mixed quality is undoubtedly an incredibly hard technical problem but I think it's probably better to sometimes let users leave Google properties than feed them inaccurate answers inaccurately attributed...
Not from a user search perspective. That's an awesome search feature. And for all the website that aren't trying to monetize, like my personal blog, or wikipedia, it also isn't a problem, and a great feature to have, which might be argued serves the greater good and improves the internet by making finding free flow of information easier.
I think we need to go back a little and think. When the internet started, if you wanted to monetize your website, and attract users, what would you do? There were no search engines. Nowadays, people claim they're at the mercy of Google, but it's more that their very existence and business model was enabled and made viable by Google. Since it seems, without Google, no one would find them and visit their website. In that sense, I find it hard to say Google is taking things away, it seems more like Google is giving and giving, sometimes, it gives less, and sometimes it gives more.
A bait and switch would imply more of an implicit contract than there ever was along the way.
In the 90s, people were making websites because they wanted to make websites, and some of them were informational, and some of them were just whatever cool thing somebody decided to publish. Keeping track of your favorite sites was harder, but eventually browsers added bookmarks and it became a bit easier.
Okay, now how do you find all of that stuff? Three answers kind of sprung up: sites that are list of sites, directories (Yahoo) and search engines. They kind of work, but finding good stuff is still hard, and while the best answer here is probably search, the indexes aren’t entirely comprehensive and the algorithms aren’t good at sorting good stuff from bad stuff yet, they’re too easily gamed.
Google came along and went with search, but they did two things differently: they built a comprehensive index and they had a good algorithm for sorting good stuff from bad stuff. This didn’t stop an entire industry from springing up to try and game it, but they made gaming their algorithm an increasingly expensive proposition, and just doing the right thing easier.
So now we have a good search engine with a comprehensive index, now the web has a different problem: as much stuff as there is, it turns out there’s not enough stuff about a lot of stuff, and this is obvious once you have a good enough search engine that will till you whether there is enough stuff about your term. Well as it turns out, it is already 2004 and people are just now figuring out how to make lots and lots of money with this internet and web stuff. Google has come out ahead of the pact as the obviously superior choice for searching all this stuff, their competitors get a lot better, but Google stays ahead of them, and they’re able to maintain a lead by simply being so much better there is no reason to stop using Google so long as they’re not evil.
They also figured something else out: people are looking for information, or the tools for getting that information. Who knows more about getting information than peoples whose jobs it is to find and surface the right information? And it turns out that some things you can’t find with just a webpage, you need other kinds of tools, like calculators. So they built them in, because at the end of the day, it’s the information that people actually want, they don’t need the debris that comes along with loading a full webpage.
tl;dr Google is an information services company and always has been. Characterizing them as a search engine or advertising company is a massive understatement and was always inaccurate.
I miss having a good index site. Yahoo's gone, DMOZ shut down. We need a good human-curated index that's not bound to ad spend or SEO nonsense. I don't want to know about "long-tail keywords", I want to know if the site has good content that serves eithet a need or an interest. Now, if it so happens that websites with valuable content also have SEO valuable, that means the search engine algorithm is optimized correctly. But we need humans.
A bunch of those things aren't really searches though, are they?
I do queries like "2 tbsp in tsp" or "45 * 22" or "$25 CDN in USD" all the time and I don't think of them as a search.
Same goes for my Echo. I ask it all the time for today's weather or what time the Texas Rangers are playing or who is the starting pitcher for the Giants tonight. None of that feels like a search.
Well, simple math isn't, but think about your currency conversion, your weather lookup, or your sports schedule. Where did Google get the data for those things? Does whoever provided that data have expenses, and did Google compensate them in any way for those expenses or did Google lift the data for free with GoogleBot?
A good comparison here is WolframAlpha, which can also answer all those queries (I was a bit surprised that the Texas Rangers example worked, but it does [1]). Most of their data comes from a curated list of primary sources, not Wikipedia or random websites.
Not only are the sources of WolframAlpha curated, they even go the extra step to make sure it's correct. From their FAQ:
"How is Wolfram|Alpha's data checked?
We use a portfolio of automated and manual methods, including statistics, visualization, source cross-checking and expert review. With trillions of pieces of data, it's inevitable that there are still errors out there.
[...]
How is real-time data curated?
Wolfram|Alpha effectively checks real-time data (such as weather, earthquakes, market prices, etc.) against built-in criteria and models. If an unexpected deviation is found, Wolfram|Alpha will normally indicate it, for example by showing lines as dashed."
I think the best one I've seen was when Google's natural language processing mistook the subject of a sentence. Rather than saying that the Mariner 10 space probe used a gravity assist on Venus to reach Mercury, it instead said that Mercury received a gravity assist from Venus.
I think they generally pay for it. It came up with song lyrics a couple of weeks ago, where Genius accused Google of scraping lyrics from Genius, but as it turns out Google payed a third party vendor LyricFind for them, and LyricFind pays the copyright holder, so if anything it seems Genius was more in the wrong...
How was Genius in the wrong? They were not paid for the service, and the data was coming from them. It is possible LyricFind stole the lyrics from Genius, but then Genius is still not in the wrong to sue them for scraping their site.
So LyricFind says they licensed the lyrics from the publisher and then verifies against other lyrics websites. Somehow, they ended up with a copy of the lyrics as posted by Genius.
Genius doesn't own the copyright to the lyrics and screwing with the punctuation doesn't create a new work, so I'm not sure they have much of a case against anybody. Maybe they could come up with a ToS violation or CFAA case against LyricFind?
And that's part of the question: If Google's getting all it's data from first party sources, say, scraping it from the team's website, it may not have harmed anyone there. But if it's scraping it from a site which aggregated and normalized the information into an easily parse-able format (hence, providing value, of which Google took advantage), then they must be compensated for it.
And in the lyrics case, it wouldn't have helped because Google wasn't scraping the lyrics. Google was paying a third party that copied the Genius site.
I agree here. For the median search I make, having to click on a website is a failure. If I search "John McCain age at death", why would I click on a website? This seems trivial, but it is frustrating for even more technical issues. If I search "Non-standard evaluation in left-hand side of mutate command R", it's a bit weird that I'm expected to click the first StackExchange rather than getting a canonical answer excerpted. Ditto when I search "Analytic standard error of a regularized regression". These are basically true/false questions, and it's a win for Google if they give me the answer.
I do emphathize with the companies being scooped by Google, and so I have concern that they are viable economically and that Google compensates them for getting the info from them.
But in the mean time, it's a success for me when my search gives me the answer immediately instead of two or three rounds of indirection.
> If I search "Non-standard evaluation in left-hand side of mutate command R", it's a bit weird that I'm expected to click the first StackExchange rather than getting a canonical answer excerpted.
Are you saying that there's no excerpt at all above the search results, when you expected one; or that there is an excerpt, but it's not from the most appropriate/relevant source?
It's annoying when there's a page that has all the information you might need, but to find it, you first have to scroll past a bunch of Stack Overflow posts that are more suitable for dilettantes who prioritise trying a quick fix over learning best practices. But when you do make the effort to find the most promising link and your browser allows the search engine to record your choice, you at least get a vote regarding how the ranking should be adjusted. When users just look at the results but don't interact with them any further, that feedback vanishes.
A win for Google isn't necessarily a win for Google users.
Try a local news site with Firefox's Reader Mode. If it's still garbage then the local news site isn't worth any more time whatsoever. Keep a running mental list of what sites not to click on. They're typically in the top 1-10 links of search results.
I wish I could hit reader mode before the page loads so my phone doesn't melt in the time it takes for the page to finish loading its 10000 js frameworks.
no dark patterns, loads quickly, uses fathom for analytics so no tracking. There are affiliate links to LendingTree, but they're just text links and i tried to be non-obtrusive to the ux.
It's interesting how one of the best sources of information are sites providing the information as a side dish to their core business.
For example I was searching for a lease termination letter template. There are sites dedicated to documents, letters, forms and templates. Many of them are pretty spammy or messy. Then I found a page of a real estate portal. They had just what I needed, without superfluous ads with nice UX. It's in their interest to help people to do this, because then they will probably search for a next apartment. They, of course, are fast to offer additional services on top like searching for the move services. But nevertheless, providing this service is something on edge what they are doing - a marketplace for real estate. Only then they will get some money, so the side experience should be as smooth and painless as possible.
Another example is a tax/salary calculator on a job board. Or... most what Google is doing to some extent, but with different trade-offs.
I know it's nothing new, but it was interesting for me to notice it recently so directly.
Don't know about everything else, but Genius is unparalleled for lyrics to rap and hip hop. I honestly don't see Google having clout to get Zack Fox to do some commentary; Tidal and spotify pretty much cornered that.
It is but Google will only put you in the carousel at the top of the search if you use AMP. That's a significant enough driver of traffic where publishers can't afford to lose it.
it's not just the carousel. it's organic rankings too.
we had an emergency AMP project after our traffic dropped 35% due to a smaller competitor implementing AMP and google rewarding them by boosting them in the search results.
That's depressing, but I will remember that when I come across an unnecessarily AMPified URL, I shouldn't blame the publisher for making this possible in the first place.
I'm using software that allows me to load userscripts in Mobile Safari on my iPhone.
I guess you'd call it an "extension"? It feels weird to use that term, as it implies (to me) that there's some sort of extension framework in place. There isn't, it's just code injection.
If the right investments were made, every software company's product could be a feature of Microsoft Windows, and Microsoft Windows could be a feature of Intel CPUs.
It's not the same as copying the "feature", in many cases Google straight up takes your content and puts it on their homepage. The difference between a product and a feature is nothing. Every app made is just a bunch of features put together. AWS S3 is a great example, it's just like Dropbox for nerds.
A lyrics website recently put fake lyrics on their site to prove Google was copying, and sure enough the fake lyrics showed up a couple weeks later.
Google isn't just taking content from "evil" companies like Yelp, they're doing it to everybody.
Job search, shopping, song lyrics, news, and who knows what else, is all being somewhat blatantly lifted. And nobody can stop it because blocking Google is a death knell to any site
So it's a case where the webmasters get greedy, and google steals their food, and everyone cheers for google. I 'm starting to think we need search engines to go back to their beginnings, when they were crawling the web , not creating or replacing the web. If they are only pointing to the websites, but rank them by speed, then webmasters would compete who will answer the question faster.
I'd rather just type CAD to USD.. or BTC to USD, or any other combination in my address bar than type xe.com, load a bunch of libraries, including Facebook connect.
Ghostly reports 6 trackers blocked.
Furthermore, it took 3.45 seconds to load xe.com in Incognito, while google only took 1.1
For an increasing proportion of users, the "address bar" is now the "Google search" bar. You may have discovered xe.com when the distinction was much more obvious, but as time went on the browser UI underwent changes, that will change as well.
New versions of Chromium will begin hiding the full URL entirely, probably so that more and more marketing/targeting related UTM parameters can be jammed in without the user ever even knowing.
I don't know about Google but DuckDuckGo pulls conversions from Xe. It is much quicker to duck the conversion than to find it after hitting Xe's homepage.
Song lyrics really ought be from a rights holders site and not j random site with questionable translations - I have seen some lyrics sites that totally mangle the meaning of basic English words.
I set up custom search keywords in every browser I use for this, among many other custom searches. In that particular case I query xe.com from an address bar search (not affiliated with them I've just always used the site) as xe <amount> for my most common currency conversion and others like xeyen for Japanese rates, xegbp for British Pound, etc. Takes me right to the page.
Even for various Google services I set them up to save unnecessary clicks.
It's not incorrectly rounded. It's correctly rounded to two decimal places in either case. Two decimal places just isn't enough digits when you give it 1 USD, so you have to give it 1000 USD.
The great thing about the calculator and specific DSLs like the exchange conversion is that it's trivial to parse. The helpfulness of the "more than just a search index" degrades rapidly into siphoning people to top 50 sites, like scraping metadata from IMDB. I'd much rather they gave me the ability to restrict the information they show me as most of it is distracting to me and reduces my productivity.
If I want to look up an actor I'll just add 'IMDB' to the end and get a much better experience on the site itself than google could ever offer in their search page.
>If you go to any sites that provide these sorts of things they're typically "scummy". Lots of ads, lots of Javascript, lots of dark patterns to make you load more page views
Exactly why go to those sites when you can get the same experience without leaving Google? And its not like Google isn't serving the ads on both Google and those "scummy" websites anyway, and no one is going to out dark pattern Google.
As if Google is not the scummiest service out there. Just because it hides its tracking and doesn't quite use dark patterns but manipulates much the same doesn't mean it isn't any better.
The presumption here, as I understand is, is that a zero-click search happened because Google used one of their inline apps to answer a question. (For example, if I search "population of new york city", I get a big and bold answer along with a graph that shows NYC's population over time, as well as LA and Chicago.)
But I'm not convinced that's necessarily true. I frequently search for things to get a general idea of what is out there, and I only skim the results. This is also something I do when I'm refining my search terms. It may take me several iterations to refine my terms until I find something I want to click through to. In this process, I may have three or four no-click searches before I land on a good query, and then I start clicking.
I've also had several fights with google over my refinements.
If I search for 2 words, odds are stupidly high that most of the results will NOT include one of my two words. Thanks Google, for deciding I didn't really mean to type that.
I follow up with added quotes, etc to enforce what I actually typed, so there will be a lot of Google-induced churn.
Google has focused so highly on the peak that more and more searches - certainly the majority of mine - have been relegated to the long tail.
DDG is worse, sadly. Unlike Google, they don't tell you they ignored part of your query. And for Google forcing a term is a strong signal and it will show to in the snippet most of the time. For DDG forcing a term is a weak signal and it will often not show up in the snippet and sometimes not at all.
I love DDG because of their bangs, but their "let me tell you what you actually wanted to search for"-attitude is pissing me off.
It is. At least google still provides "verbatim" mode, which is annoying to enable and still doesn't work some of the time, but DDG has no alternative. So if I search for "mispselling" for example, with double-quotes around it, 0% of the results contain my word and DDG doesn't even mention that it altered my search term. I know this is a bad example, but I've encountered it in the past with legitimate searches where I need results for some acronym or product whose name is close to another common word.
Another factor is snippets. They grab a sentence or two from the web pages that are your search result, and display that inline. Sometimes that gives me all I need to know. It's debatable whether previews of a web site are really a walled garden. (If I like what I see, it drives me toward the external web site.)
However, while I agree that there are other factors (and we shouldn't make assumptions), I do think the walled garden effect is real. I find it particularly annoying on Android when I ask Assistant for something and instead of showing me search engine results and a Google inline app, it shows me just Google's answer and has a button to do a search. Adding an extra step to see non-Google information feels like they're trying very hard to steer me toward their stuff.
Google’s currently helping to perpetuate the myth that New York is a growing city and its population is at an all-time high of 8.623 million. In reality, it’s a dying city, and the current best estimate is 8,398,748; every bureau except Staten Island is losing population. People are fleeing New York (both the city and upstate) due to high taxes, crumbling infrastructure, bad weather, and a generally poor standard of living.
"Each of the city’s five boroughs registered gains in population. The Bronx saw the largest increase, up 3.4 percent, followed by Brooklyn (3.1 percent), Manhattan (2.7 percent), and Queens (2.2 percent); Staten Island showed the smallest gain (1.6 percent) over the 99-month period."
The slight decline has been happening for two years. You can’t state a bunch of reasons over something that just started. As if you know those are the reasons.
Sure, but "nuance" and "context" is not what Google is trying to optimize for. They are going for the "instant fact check" audience, like the sidekick gophers on podcasts who look stuff up while the main host and their guest warbles on.
The parent included their biases no proof agenda in their comment. And that seems to be the primary reason for the comment in the first place. I’d say that’s worse than Google.
I cited a government website showing that the population is off by over 200,000 people. I don’t know if you’ve ever lived in New York, but it’s full of world-class bullshitters, so much of what you hear about New York is bullshit. Bill de Blasio claimed there were 2 million people standing in Times Square for New Year’s. How in the world could that possibly be true?
I do understand that people want clicks to their site so that they can get ad views, and that those ad view payments are what pay the hosting bill and reimburse for the effort of generating the info.
Yet from a UX PoV, I simply want to know the airspeed of an unladen swallow. I don't want to click to learn it. That additional friction doesn't improve my user experience, it improves someone else's. Mandating that would turn Google into a splash screen or landing page, and we've gotten rid of those anti patterns for good reasons.
I chose my toy example (unladen swallow) because it's an example where nuance is required (African or European?) and if google's answer box were to obscure that it would be wrong. But if answers that with a "two answers: African x European y, and click here to know more" that is a user-friendly model.
Really objecting to google doing this is like objecting to an ad blocker IMHO. Ad blockers upset sites, but make users happy.
(anti disclaimer and disclaimer: I am not a fan of google, use DDG, and my company has ditched all google services. OTOH my gf works there, though not on anything related to search.)
The problem is that people may stop making sites that tell you the airspeed of an unladed swallow if Google's snippets mean that they never get any visitors on their site. Then Google search won't be able to help you because there aren't any sites to copy snippets from. As for the comparison to ad blockers, I think the situation is different. I doubt people install ad blockers just because they don't like ads; they don't like the slow websites, privacy, and security implications associated with targeted ads. Search snippets are punishing sites just because they aren't Google.
Indeed, that was kind of my first point. I don't know the viability of a model that requires users to jump through an extra hoop (click and wait). That's the "splash screen" or "landing page" problem. I also wonder how many of those sites are really making any money even before ad blocking.
FWIW I block ads because they are intrusive, as well as for speed.
On the other hand, your search is reducible to a single 'answer'.
To make a counter point - I was recently researching 'things about various electronic and motorized bicycles / scooters'- a subject which a 'good answer' would be some sort of curated content by a specialist individual who is interested in the topic or a specialized vendor who has lots of 'information pages' - but instead of finding much of this at all, nearly every search went to various amazon / alibaba / ebay pages with very little non-commercial general purpose content available.. (not even manufacturer pages were there really)
and 'conveniently', most of the 'organic content' these days is non-texual and tied to youtube, or hidden on non-indexed facebook groups..
net result: a network effect whereby walled gardens drive organic content off of indexes, indexes become more crowded with commercial content, and then commercial content has to be paid to get to the top of the list to get traffic, and one can go to youtube to be user profiled for targeted marketing - how exactly did my purpose-specific container tab yield targeted ads to a completely separate yahoo mail account again??
Indeed, I don't think these boxes work at all when the question does not reduce to a single answer. Does google supply them anyway?
I did recently do a google search for someone's name and felt like I was presented with an anti pattern: a couple of boxes not getting to what I wanted (and not the wikipedia link) followed by a row of small boxes with thumbnails of other people goog hoped I might be interested in, a couple of more large boxes and then finally, after scrolling up, the text links. Yuck, as bad as altavista/Yahoo/lycos.
> That additional friction doesn't improve my user experience, it improves someone else's. Mandating that would turn Google into a splash screen or landing page, and we've gotten rid of those anti patterns for good reasons.
Long-term (potentially): those sites no longer exist, so there's no way to learn the information with or without a click.
* Ads have tracking and security implications. For clicking though to a site, that is far less the case (an individual site can't correlate my visits across multiple sites, whereas an ad network can).
* Going to the site at least gives an opportunity to present alternative monetisation strategies (direct payment or donations, or selling additional content/services - depending on the nature of the site). If you are just getting the information from the Google page, that disappears.
Well I've hit websites that break completely with ad blockers enabled. I've also seen sites that break regardless of my addons, just because it's Firefox, and it thinks I've installed something that's blocking ads specifically (frankly it's offensive to tell the user to disable something they never enabled, while blocking them).
Meanwhile, I waste my own time looking at how these are implemented because in about half the cases (or so) I can simply "inspect element" and delete the offending code. Though I waste the time in the other cases.
It's a disgrace, and something we shouldn't stand for...
More seriously, from a moral standpoint, I would argue that what's at most a short paragraph extract from a website falls under fair use. I don't believe any site is entitled to clicks from Google's search results page, certainly not for such short snippets, which I believe don't exhibit the level of creativity that deserves denying the right to copy to others. By contrast, the other sites you listed provide copies of significantly longer and more creative works.
A dictionary is a collection of short snippets. I find it hard to believe that showing those snippets for whatever word you search falls under fair use. The people doing the hard work of making a dictionary deserve to serve those definitions under their own terms. Or at least some combination of their terms and the user's terms, instead of just google's terms.
> The people doing the hard work of making a dictionary deserve to serve those definitions under their own terms.
This depends on the country. In the EU, this may be the case. However, the US Supreme Court rejected the "sweat of the brow" copyright doctrine in https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... For a creative expression to be copyrightable in the US, it must have some minimum threshold of originality.
The wording of dictionary definitions themselves may be copyrightable if sufficiently unique. But the question of fair use should be considered as well. If Google as a matter of policy always consulted a particular database for a particular class of searches, without permission from the copyright owner, perhaps one could argue that crosses the line of fair use. But for isolated results, it doesn't appear much different than the current practice of putting snippets of the page below the links in the search results. For more extensive databases, Google often has an explicit agreement with the database owner.
Disclaimer: I am not a lawyer, I don't know anything about the implementation of the answer box, and certainly don't know the law in multiple jurisdictions.
It sounds like you're looking for legal loopholes. Which is absolutely fine for a business to do. Just don't use phrases like "from a moral standpoint..." when you really mean "from a legal standpoint...".
I don't believe Google has a moral obligation to send traffic to the sites that appear in its search results, any more than a book has an obligation to be bought or borrowed from the library by the readers of a short quote from that book in another book. So, for me, it is a moral question as well.
> don't believe Google has a moral obligation to send traffic
Bizzare. The only reason google gets traffic is because of those sites. Your comparison is also off, google is not another book (they dont create content), it's the librarian telling people which book to borrow.
In your analogy of books, google is a book too. If google as a book dictionary was just a copy of your book dictionary, that would be illegal and wrong. You the reader are only looking at a single definition, which could be a plausible fair-use quote. But when google's entire dictionary is just fair-use quotes of your book, then they've just copied your book.
Indeed, one of the four criteria in determining fair use is "how much" was copied.
You know what another one is? Whether the usage affects the originators ability to make money from their work.
A legal determination of Fair Use is subjective and can only be decided in court. But, citing only one of the four pillars as moral justification while blatantly ignoring the other (equally important) ones is a little rich.
I agree that there are other factors to be considered, including the ability of the originator to make money from their creative expression. However, from the page you linked, "[b]ecause the dissemination of facts or information benefits the public, you have more leeway to copy from factual works such as biographies than you do from fictional works such as plays or novels." In many cases, Google's knowledge box comes from sources such as this. In the US at least, given the short length of most snippets in the knowledge box, you'd have to argue that the particular expression of the fact in the search box is both sufficiently creative and sufficiently central to the work. Even in the case of a snippet reflecting human judgement, such as the box displayed for "best nail for cedar wood", the short snippet appears to be considered fair use, at least in the US. Google Books has been considered fair use, for example: https://arstechnica.com/tech-policy/2016/04/fair-use-prevail...
In the case of artistic works, the criteria for fair use outside of parody are more limited (although see the Google Books case). For example, the display of a stanza of a poem, or a section of song lyrics, might not be seen as fair use (I'll leave my personal opinions out of this). However, in the case of song lyrics, Google has permission from the rights owners to reproduce them in their search results.
Dictionary or acronym definition sites which cram their preview with a ton of useless boilerplate (most of which also litters their page) is a growing Modern Frustration.
Most of the time I've got dict or some hand-tuned extractor scripts which can pull the information for me, though if I'm away from those the experience remains annoying and convinces me to actively seek down the page for the first result which does NOT do this.
Precisely, and in this mode Google is functioning as an (ad-supported!) ad blocker. I can see a conflict of interest here even if google's ads are less intrusive and more easily blocked than the dictionary sites'.
Article should be: Avast antivirus users click less than half of google searches. Their data isn’t representative of the market broadly. For one thing Microsoft claims bing has 9% market share, while avast pegs it as 1.4% market share. https://mobile.twitter.com/MSFTAdvertising/status/8982080475...
> That includes millions of Android-powered devices and millions of PCs as well. They have much lower coverage on iOS devices and so don’t report on visit data from those browsers.
I'm not saying that their sample is not heavily biased, but they do claim that it's not a desktop-only dataset.
My mistake. I didn't realize anyone was stupid enough to put an antivirus on an Android. Please amend my complaint to say that the sample is biased toward morons.
I have a particular page on my blog that mostly appear as the answer to a question "Can you do X with Y tool". Before Google started displaying it as a snippet, the post was getting decent traffic, and the comments were more somewhat moderate.
Then the snippet appeared, and a bunch of negative comments started to appear. To clarify, my answer was not a solid yes or no, but more of my experience with the tool. But Google snippets conveniently answers with a section that says: "It is pointless."
User's read the snippet, are enraged, click, don't bother to read the article, jump to the comment and say "YOU ARE WRONG."
Now, the controversy brings a decent amount of traffic. But unfortunately, the traffic is not from people reading in the first place.
Article: "Throughout this post, I’ll be using numbers from the clickstream data company, Jumpshot. They are, in my opinion, the best, most reliable source of information on what happens inside web browsers because of how they gather, process, and scale their estimates."
Me: How do they get this data. This is some pretty thorough data on a site they don't own.
Article: "Jumpshot is the data arm of Avast, a well-regarded maker of numerous Internet security products. This suite of products, in order to function, must collect and analyze every URL visited by every browser of every machine on which its installed. [...]
Article: "Because Avast has to see and process all these URLs anyway (in order to serve their function of providing web security), they anonymize, aggregate, and remove any personally-identifiable information from the browser URL visits and then provide them to Jumpshot, who then makes estimates about broad web usage behavior."
...I hate this world. Everyone's tracking you. Everyone's selling you to advertisers.
For those who skipped the article, this data is exclusively from users who installed Avast "security" products, which apparently are sufficiently invasive that they get this sort of data and sell it to third parties.
The article thinks this dataset is close enough to representative to present their conclusions as more or less facts. I'm not sure I'd make the same conclusion.
I trust Microsoft and avast to not lie about facts as they can get in big trouble for that. Rather I don’t believe this data from avast is indicative of the market broadly.
Does anyone else use search at least a dozen times a day just for spellchecking a word or confirming a phrase? I find myself pasting phrases into the search bar when autocorrect or spellcheck just doesn't do the trick. I'd love to know what percentage of "searches" were never actually looking for a target.
Yes, absolutely. This is very common strategy among folks whom are working with different languages. Also looking for confirmation in "reverse dictionaries" because Google mistranslates some words.
The problem seems apparent to me. Google has a monopoly on websites being able to promote themselves.
There are plenty of other channels, but none with remotely the reach or trust of the US's overwhelmingly dominant search engine.
By dint of this fact, they can steal the content of those who actually produced it. What are you going to do, deindex your site? That's death, because Google is a monopoly.
In the short term it seems good for the end user. But in the long term, Google having a monopoly on distributing the world's information seems like an obvious threat.
If I search for something using Google, and don't click on a link, doesn't that mean my search was fruitless? What else do we mean by 'zero-click searches'?
Edit: Unless it's the simple '6 x 10' or 'What is the time in London', in which case, are we surprised these queries are handled by $SearchEngine? DDG does the same thing (albeit, uses a third party website and gives credit)
Example query - “Leonardo DiCaprio‘s net worth”. While previously this would have resulted in a click to the top website that put in the effort to compile this information, now google would highlight that exact sentence with the answer right there. The site loses a click through and any potential revenue.
> The site loses a click through and any potential revenue.
I'm having trouble understanding how the site would have lost potential revenue. All the query wanted to know was the net worth of someone. I don't imagine anyone casual enough to use Google to answer that who would be willing to spend a lot of time trying to figure that out using an advertisement-laden "free" website. I don't imagine anyone professionally interested in that answer who'd use Google as their primary resource nor would they click on ads on the target site.
On the other hand, Google gets to know who's talking about whom. That in and of itself is/can be valuable. I can see an argument be made for abuse of monopoly here. On the other hand, I think what Google does indeed benefit the consumer in this case. The casual user's experience is likely improved and the professional user's experience still wouldn't use Google as a primary resource.
I'm having trouble understanding how the site would have lost potential revenue.
They lost ad impressions.
I don't imagine anyone casual enough to use Google to answer that who would be willing to spend a lot of time trying to figure that out using an advertisement-laden "free" website.
Why not? It's not like that they had any other options than trying to extract that information from ad-laden websites. Most people read rather slowly, and by the time they either find out what they need or decide that they give up, lots of ad impressions have already had happened.
Sure, that counts as revenue. It's not something I'm going to lose sleep over. I don't exactly think that advertisements should be a profitable "feature" of the internet in its current form.
> Why not? It's not like that they had any other options than trying to extract that information from ad-laden websites.
You completely missed my point. Now that Google "owns" the data and provides the answer, the user won't need to click on a scummy site.
Moreover, casual questions not answered directly by Google are answered by casual browsing. Casual browsing happens with ad blockers. If a site breaks when an ad blocker is in use, users exit the site.
We do, though the counterargument is that sites that currently publish content like that will have no incentive to keep doing it. If that proves true, presumably in long run you won't be able to get good answers to similar questions, which I suppose could force Google to create their own content, a sort of search-engine version of Netflix and Amazon's move into film-making.
> If I search for something using Google, and don't click on a link, doesn't that mean my search was fruitless?
No, it could mean that Google itself provided an acceptable answer for you in an infobox.
For example, if you search for "San Francisco temperature," the top-of-the-page result is an interactive box showing the current conditions and 24-hour forecast for the city.
That's reasonably likely to answer the implied question without redirecting you to another site, so it is an example of a successful "zero-click search."
Google scrapes the info and then displays it on the search page, meaning you get your question answered without needing to click through. So basically denying anyone else traffic, while still being useful to end users.
Google can preview results for clicks by exposing a site's content in the search results.
E.g if you google Obama you will see a wiki blurb about him on the right
I mean that’s could be one explanation for no clicks. But if that would be the primary cause it would imply google’s results are getting worse, which would lead to someone else increasing market share. The primary reason is that google shows you information scraped from other sites directly on the results page so you never have to click on any links for a lot of queries.
If you're searching for a quick piece of information, that might be included in the description text, or one of the cards that google might generate for your query (like when doing a calculation, unit conversion, weather query, stock ticker query...)
How many times does your first search query result in a page you want to click on? It usually takes me a couple of tries for anything interesting I'm looking for.
As another poster mentioned, it seems logical that some reduction in click-thru-rate happens because of Google's "Featured Snippets"[0]. The Google search results page shows an extract of a web page that likely answers the web surfer's query. We don't know what % of zero clicks was caused by the snippets info box.
To be fair a lot of the time I can get the info I need directly from the search.
- I use Google as a calculator and a unit of measure converter including currency
- I don't hesitate to "define WORD" when I don't know a word which gets displayed at the top
- What is the capital/population/square miles of
- Who is/cast of/CHARACTERNAME actor
- What muscles are used with movement/machine
- synonym/antonym WORD
- SONG lyrics/"turn off your mind relax and float down stream" song/'Tomorrow Never Knows' band
Actually, every single time I open a Google tab I query "define define" first so that I have the search bar at the top and not in the center of the page. Don't judge me, you do weird stuff too.
And with AMP, Google is trying ti do the same with entire sites.
In the long run why will we continue to produce content for Google?
Google must stop trying to suck the hole world into itself. Google is starting to look like a black hole.
And the "it is good for consumers" or "users like it" are just offensive. It's good for Google. It may be good for users today but it's bad for content producers and in the long run it's bad for everyone.
All the other major search engines also use snippets and AMP precisely because users want it. Should all of them stop too? Can you clearly define the regulation you are proposing to impose on search engines?
When I read the title I thought it was going to be an article praising Google, but the author seems to actually think less clicks is bad?
I remember back in the early 2000s when I had to click through 1st to 10th result trying to find what I was looking for, sometimes even having to go to 2nd page of results.
Now I use search for everything : name of a built-in function, quick calculation, checking the spelling or wording of a sentence, short biography of someone... For none of that I will need to click on the result because the answer is given to me without needing to click anything.
The correct answer in SO questions is always the most voted one, not the accepted one. I always check for that. I think I've seen SO recently sort the answers of a certain question by number of votes by default but I might be wrong
It feels to me like the SEO optimizing folks have gotten out ahead of Google's algorithm. I've been getting back into an old hobby and looking for where to buy supplies or advice on which supplies to buy and it's just all spam, fake sites and low quality products.
And it occurs to me that I've been experiencing a lesser version of this for years. Google search just isn't working for me anymore.
Lots of people in this thread claiming that google is slipping, focusing on the wrong metrics, etc. But I agree, google is losing the battle against SEO and its a really hard battle to win. Eventually there will need to be a shift away from search as we know it to win this game.
Yes, this is also serious threat. SEO and spamming affecting quality of search. Some years ago you could just type some scientific article's title and it would have popped up good references and maybe the pdf version. Even you could find complete books like that and a little of refinement. Now it's all spam, at least for the first pages.
As bad as this sounds, I got to say I'm not even mad. The instant answers Google provides seem to come from websites that are mostly unusable, bloated with tracking and other BS that increase load times, and even worse SEO-content (that stuff that repeats words just to have more "relevant" keywords). I don't feel too bad about those pages losing traffic.
Longform content, interactive content, blogs, videos, forums and social networks (generally pages you might want to spend more time on) seem to be mostly unaffected.
If Google kills CelebrityNetWorth's revenue, Google no longer can scrape updated data from CelebrityNetWorth, because they killed it already. So Google data gets worse or remains static, which quickly becomes inaccurate.
The EU is attempting to fix this issue with the so-called "link tax" everyone has claimed is horrible for the Internet, but really, it's a way to correct the industry. For Google to have good information to scrape, the content creators need to be fairly compensated for that, or else the whole thing falls apart.
Because right now, what Google has done, is acted as a parasite, but a poor one, that actually kills its host.
Instead of an impossible to implement link tax, how about we just made it illegal for search engines to provide these kinds of "pirate" snippets without an opt-in... Either show meta desc, or don't show it at all. Adjust results by relevancy and pagerank (as they used to do) and everyone is happy.
There's nothing impossible to implement about it, but the marketing from the opposition has been really strong. It isn't even a link tax, for example, but it's detractors have pretty successfully rebranded it.
> Maybe I need to stop being disappointed in Google and just start expecting them to lie, mislead, and refuse direct questions
Why exactly is it in Google's best interests to reveal internal competitive trade secrets just because someone asked? Or any company, for that matter. If a company discloses information it's because they think they'll gain more from disclosing it than from not disclosing it. So why should you demand that all companies answer all your questions?
Especially now that organic results often don't even show up without scrolling down. I miss the good old days when being the #1 search result meant something!
What I find truly disgusting is how, as a business, you basically have to by adwords against your exact domain name, lest competitors buy it and show up above your organic positioning in search results. Can't think of a better example of a "Google Tax".
Somewhere along the line Google stopped being about finding a website and instead is about answering a question with the answer sometimes being a website.
One could argue they have always been an answer engine and they are just getting more direct and precise.
I think another decreasing search type, going by my own example, is the programming syntax/method name search.
If I can't remember exactly the syntax I can often construct a query precise enough that the first result shows enough of the syntax in stackoverflow's accepted answer that I can just proceed from there without clicking through.
2. Preference for more established / larger / more popular websites in the "organic"/non-ad results. This is something I can't prove, but my and my acquaintances' experience suggests that, gradually (mostly before the last 5 years, i.e. not a new thing by now), Google has shifted toward tending to direct you more to those kinds of sights. Perhaps others have studied this.
~45% of google traffic is "organic" - I search, I find a link I leave google
~5% is as clicks (!)
~50% is zero click - it's common enough a query that google just answers it right there, flights, trains timetables, cinema, exchange rates
This is kind of a huge shift and also kind of not important at all
Millions of us want, say, cinema listings, and off the back of those listings might easily buy a ticket. I can see this as a huge business for Google and certainly disruptive but I am not sure we are losing much by Google vetting it's dollar instead of the same dollar going to the ticketing company behind a multiplex
Is this the old fear about microsoft - that all any ISV was doing was market research for microsoft?
If you are a distribution channel selling a barely differentiated product to millions of people - well the writing has been on the wall for a while.
This will likely backfire in the long term. People writing content will shy away from writing articles on subjects or which answer questions that google will simply steal and display. Eventually the existing sources will become out of date and google will begin to serve inaccurate information.
That content will be much much harder to extract and determine authority than what google is using right now, giving greater chances of it being wrong.
For me, the issue is not that search queries can now answer simple calculations or inane celebrity trivia without requiring clickthroughs to ad-ridden sites.
It's what comes afterwards, when Google inevitably starts answering queries on complex issues around medicine, politics, society or the economy with a pithy snippet grabbed from what it's machine learning thought was representative.
We've already seen that vast swathes of the population can have their vote influenced by Facebook/Twitter posts with a big "Sponsored" tag on them. No nuance, all invective.
How do you think that will play out when people will take the snippet at its word and not bother to at least click and get some idea of the author's possible biases, methodological limitations, etc.?
Not surprised, but the watch out for Google investors is that all the tricks for Google revenue growth to outpace general internet growth are all played out.
You can't "add more ads" above the fold when everything above the fold (for lucrative searches) is already an ad.
And dropping below 50% of searches don't result in a click is hard. Either for practical or anti-trust reasons.
So, get used to growth that's tied to internet growth. Which "ain't so bad", but also "ain't what it used to be".
This cow is milked, so if you want to go back to growth that exceeds expectations, you need a new cow. Search monetization has peaked for the near future.
A less techie theory: Witness the rise of the Google Skeptical Search user. In essence, Google search users maybe developing an attitude of skepticism towards the quality and usefulness of the results they receive. It started with news searches and its spreading to everything else search related. The sensing being that Google's search results are not an honest reflection of the information that is available.
It's amazing how they can get away with this, while at the same time not allowing others to scrape their sites. The actual content creators probably see significantly less traffic because of this - without ever knowing what the actual traffic could have been... there should be some kind of pay it forward/revenue sharing that Google should do. But that would be not being evil.
> This suite of products, in order to function, must collect and analyze every URL visited by every browser of every machine on which its installed. ...
That's why these products are privacy nightmares.
Bitdefender has admitted that it compromised a user.[0] That was a major case. I'm sure that many other users have been hosed, in less newsworthy ways.
> Because Avast has to see and process all these URLs anyway (in order to serve their function of providing web security), they anonymize, aggregate, and remove any personally-identifiable information from the browser URL visits and then provide them to Jumpshot, who then makes estimates about broad web usage behavior.
100% of my searches produced no visible(to google) clicks for over 10 years.
var resultLinks = e ? e.querySelectorAll('.r') : document.querySelectorAll('div#search .r');
for (var i = 0; i < resultLinks.length; i++) {
var link = resultLinks[i].childNodes[0];
var oldLink = link.href;
if (/^(https?:\/\/(www\.|encrypted\.)?google\.[^\/]*)?\/?url/.test(oldLink)) {
var matches = /[\?&](url|q)=(.+?)&/.exec(oldLink);
if (matches != null) {
link.href = unescape(matches[2]);
}
}
// Clear attached event listeners so google can't mangle urls on mouse click
if (link.getAttribute('onmousedown')) {
link.removeAttribute('onmousedown');
}
if (link.ping) link.ping = null;
With AMP, signed HTTP exchanges, and "portals" it's getting even worse. Users never leave their servers, and they control the publishing format and require you to include their JavaScript. (It's only briefly mentioned in the article.)
There are plenty of types searches where I'm now viscerally afraid to click on the results lest it result in a clusterbomb of pages I can't close without multiple dialog boxes shooting out all over. Ublock helps but muscle memory is long.
Funny, the article says a CTR of 4.42% is average. Yet my 5% CTR for my own domain name, Google Adwords charges me upwards of $0.50/click, and says I'm penalized (higher cost) because of low engagement.
This is for keywords with no competition, which makes me a bit bitter because I have a substantial negative ROI. I can understand that Google wants to prevent arbitrage spam, but I just last week decided to cut my adwords spend to almost nothing because it's not getting better over time.
Just remove your microdata and other site-specific metadata. HTML was fine to begin with, its not our problem if facebook or google can't read a <meta description> tag properly. Everyone forgives the transgressions that google does because "it's Google", but when Elsevier does the same thing (appropriating and locking other people's IP for huge profits), they 're starting riots.
Not to mention the issues are apples and oranges - Elsevier is acting as a gatekeeper for public research. Google is at worst exploiting fair use at scale.
And with AMP, Google is trying to do the same with entire sites.
In the long run why will we continue to produce content for Google?
Google must stop trying to suck the hole world into itself. Google is becoming a big black hole.
And the "it is good for consumers" or "users like it" are just offensive. It's good for Google. It may be good for users today but it's bad for content producers and in the long run it's bad for everyone.
The dream of AI is to build a StarTrek like computer that can read and understand all human knowledge and answer questions about it, synthesizing answers.
Web Search for factual answers is a stop gap on that road, but distributed knowledge databases are inexorably going to be centralized, melded, and transformed.
The 90s “content is king” business models just aren’t sustainable if what you’re publishing is static mostly factual content.
Any DDG users notice how it does the same thing with the top StackOverflow answer? From the user's perspective it's fantastic, but I assume it's legally at least a grey area. Considering how aggressively SO is monetising now they'll probably come after them before long. (Of course, DDG's userbase is pretty small, but probably a not insignificant share of developers.)
I’ve mostly stopped using Google search due to the amount of sponsored/summarized content on the first page, which either renders my search useless (too many references to generic pieces or reviews rather than actual product pages) or abridged snippets from web sites (that often carry just enough information to be misleading).
I use DuckDuckGo, obviously, especially if I need technical information.
I admit to using google as a spell checker. That is to say that my browser spell checks but doesn't give me suggestions on how to fix it. I would say at least 2/3 of my "searches" are really just spell checking (or small calculations). I guess that's why I get such weird ads :-D
In some cases could this be because fewer than half of searches are returning relevant results? For my part I've been having a really difficult time finding things in Google lately. Even with search personalization on, results are just.. completely off the mark.
As a reminder, DuckDuckGo (while being an alternative itself) supports bang operators (https://duckduckgo.com/bang), which helps to prevent Google assimilation.
What does "Zero-Click Searches" mean exactly? I know that when I search for "1 pound in eur" Google will tell me the answer right away, but these kind of searches can't possible account for 50%. What else am I missing?
One example is "how to tie shoes". It scrapes the content from wikihow and displays it to you. There is no need to click through to the site to see the content. Something similar happens for recipe searches.
I do this all the time. What year did this album come out? Search the name of the album and the year comes up in the sidebar. How old is this celebrity? Search the name of that celebrity and their age comes up in the snippet on the side.
Google benefits by crawling the content everyone else creates and by being the aggregator for consumers. So it’s unfair to use other people’s content for rich snippets and not reciprocate by referring traffic.
I want to take the opportunity to blame SEO for this. The top results are games so hard that not-rarely the first page is nothing but spammy seo sites now. Depends a lot on what you are searching for though.
This is Brian from the website celebritynetworth.com. I submitted the following statement to the House subcommittee on Antitrust detailing the history of Google scraping our information. The TL:DR is that Google approached us about getting an API to our data. We said no. Google scraped the whole site anyways. Google frequently displays our information directly in its results without any attribution in a large snippet that takes up the entire screen. The answer box is so large and dominating of the SERP that users have no ABILITY to click through. Let alone need. Google's actions have absolutely decimated our business. Here's our statement with image examples dating back to 2012: https://medium.com/@brianwarner/celebritynetworths-statement...
Is this data from browsers infected with Malware like my Mother-and-Father-in-Law's computer where all they do is search "the Google" for recipes or whether a celebrity is dead?
Paid Ad
Paid Ad
Paid Ad
Paid Ad
Paid Ad
SEO optimized not what I need
SEO optimized not what I need
SEO optimized not what I need
Kinda maybe what I need?
<next page>
When Rand last posted about this it inspired me to look into other channels such as email and social. Basically clicks are disappearing for ALL channels, this isn't just a Google thing, it's a digital marketing thing as a whole. My thoughts here: https://kolemcrae.com/no-more-clicks-makes-me-sad/
since the 2008 recession Google has tried to avoid sending people to sites, preferring ads and Google sites instead. Each year /update they have tightened the screws. No doubt Google execs are saying, look we still send about 50% of people to other sites, there's a lot more growth opportunity.
Maybe compare this with only reading the headline on Hacker News?
We often prefer non-clickbait headlines for good reason. The answer to a question may be simple enough to fit into the headline, but the site will withhold it to get a click.
Paywalls also encourage people to skip reading the article. I find I'm a bit annoyed at first, but it fades quickly and I realize the news probably isn't that important anyway.
I didn't see it defined on the page, so I'm wondering, do zero click searches count frustrated users giving up because the first two pages are entirely full of SEO optimized commercial websites when searches are made for informational content?
Try searching anything related to researching foundation repair. You get pages upon pages of foundation repair services and biased one pager summaries about certain procedures.
Long gone are the days when search queries retrieved educational content - when was the last time you saw results for engineering toolbox, or hyperphysics, or anything other than ads, commercial services, and stack overflow for software related questions?
SEO killed search and Google is complicit. I think this has massive ramifications for society - the sources of information are shallower, and people are increasingly unable to notice differentiate objective knowledge sources from commercial websites. The people who are interested in generating and curating true knowledge repositories are probably not interested in paying for SEO, so their efforts are effectively hidden from what amounts to the gateway to the internet.
Google doesn't search for what you ask anymore. It tells you what to search for based on a prompt.
Alphabet is the most oppressive entity in the history of the known universe.
I guess this isn’t related, but am I the only one who feels like Google search results are going down hill. Maybe 5 years ago I could find a lot of information from small and big sites. Nowadays, it feels like half the Google is just literally ignoring some of my search terms. And the other half they are explicitly ignoring them by crossing out words because they assume I search the same things as everyone else?
I dunno, maybe my queries are getting more complex? But I’m getting a strong feeling that Google is becoming less and less useful as an indexer if the web.
Ok so Google is a monopoly in this sector that's pretty clear, but what's the real problem here? Their competitors offer inferior products so we don't really have a case. Until they can develop better technology or offer superior products than Google's and still are unable to gain market shares then that's when we will have a real case.
Besides, for any business to continue leading long-term in modern days it's inevitable it must seek to become a monopoly, or at least part of a duopoly.
Yes, they link to you below the featured snippet.
No, more people don't click, because they've taken the answer from your website and displayed it right in their search results.
For example: If I'm searching for "best nail for cedar wood" Google gives me the answer: STAINLESS STEEL - and I never had to click through to the website that gave the answer: https://bit.ly/2MdovdP
• Yes, this is good for users (it would also be good for users if Netflix gave away movies free)
• Overall, the publishers who "rank" for this query receive fewer clicks
• Google earns more ad revenue as users stick around on Google longer
Ironically, Google has a policy against scraping their results, but their whole business model is predicated off scraping other sites and making money off the content - in many cases never sending traffic (or significantly reduced traffic) to the publisher of the content.