Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Why does Pinterest dominate Google text search results?
689 points by Winterflow3r on Dec 25, 2020 | hide | past | favorite | 312 comments
More and more often when I search (using text queries, not image search, which I know has been polluted by Pinterest for years), I get pages upon pages of Pinterest results, sometimes the same Pinterest page but from the different pinterest country domains like pinterest.fi for Finland and pinterest.se for Sweden. Does anyone know if Google gives Pinterest preferential treatement in SEO rankings?

Edit: A few comments were asking what my queries were to generate search results where Pinterest dominates, so clarifying that a bit. I run a site that has a colour search engine for lipsticks and since Google is one of the dominant ways in which people land on my site (searching for things like "nyx budapest lipstick dupes"), I was studying various makeup related queries to see which sites ranked highest .

Edit2: Edited the title for clarity - I mean text search, not image search




There's no intentional or manual effort at Google to promote pinterest.

Pinterest shows up because they understand how the Google algorithm works and built their website to display all the signals that Google looks for in relevant image content.

They understand user intent and generate URLs that present content in a way that google expects to see.

Examples of how they do this from their engineering team:

https://medium.com/pinterest-engineering/demystifying-seo-wi...

More:

https://medium.com/pinterest-engineering/tagged/seo


Google should be asking from a product perspective if they are providing the search results the user is most likely seeking.

I seriously doubt that most pinterest results are what users are seeking. As such, Google probably should consider changes to their algorithm to compensate (cat and mouse) for pinterest's gaming. That is assuming how pinterest does it is not against any google rules... because if it were against any rules, they should be slapped down hard.

When my searches result in mostly pinterest results, I give up more quickly. That means I spend less time using google, and that in theory means less money for google. Unfortunately, I do see other more regular user types operate their computers, and they are less or un-descriminating about the domain name listed under the image. So they'll head directly into the pinterest waste pond anyway.


Yeah, Pinterest is almost always disappointing as a result: their account requirements are annoying and it tends to obfuscate the source of the image or information you’re looking for.


A few times I've landed on Pinterest and they lock user copied (contrary to copyright law in my country AFAICT) content behind a registration-wall. They never show enough utility for new to register, they just tease on Google and then don't show what they teased.

Google results would be better for me with all Pinterest results removed.


Wasn’t there an option at one time to tell Google to hide results from a certain domain? I feel like I haven’t seen that in years though


It still works

-site:pinterest.*

Even with wildcard tld, otherwise it is still all Pinterest

But it is interesting that those kind of Google/search engine feature hardly ever get talked about these days whereas in the past they were considered crucial to get decent results.


I don't know what the pathway for this is but just -pinterest does it for me.


-pinterest removes pages that contain the term pinterest, so pages from a pinterest.* domain, plus articles about pinterest, etc.


Sounds pretty good to me. If I'm searching for a term, but don't want results from pintrest, then I'm probably not wanting to read an article about pintrest either.


"Here's a picture I took today. I also posted it on Instagram and Pintrest."


Except that the word "pinterest", like pinterest.* results, appears in all the nooks and crannies of the rest of the Web. Excluding the domain like that succeeds in narrowing the scope of the search without excluding content unduly.


If you wanted to read an article about Pinterest, it would probably be hard to search for it in any other way.


I use that a lot. Wouldn't Google notice if a lot of people were telling it they don't want to see results from Pinterest and make some adjustments to their SERPs?


That's a nice hack, thanks! I've just modified my Google search engine entry in Chrome to include this.


There use to be an easy way to configure your account to always block certain domains, but that's gone now, so you have to use -site per-query. Maybe there are browser plugins that help? I miss the old functionality. I only had to block about a dozen sites to get great search results. Maybe it'd be less viable now; I'd probably have to block hundreds or thousands of sites, and they'd keep breeding new ones.


Just make a search that excludes all the domains you despise. Don't include any search terms or just a dummy one.

Then save it as a bookmark.

e.g.

  "-site:foo.* -site:bar.* ..."
Use that bookmark as your search engine: click on it, then fill in search terms.


You used to be able to tweak part of the URL in the Firefox google search thing, and append queries I thought so as well.


Yep. Google removed it.

https://github.com/codyogden/killedbygoogle/issues/674


You can hide sites with a flag in the query. No idea if it works on image search but I assume it does.

    -site:pinterest.com


Just add `-site:example.org` to your query.


I have done this throughout COVID for the NYTimes' content, as it's behind a paywall.


Interestingly, if you use Brave, it lets you hop to the tor versions of sites, and since NYTimes has a tor version, if you visit that, it’s paywall free.


I use Brave. How do I hop to the Tor version of a site?


If you go to a site that advertises (via HTTP header) that it has a Tor version, you'll get an "Open in Tor" button at the end of the address bar.

Examples are nytimes.com and dw.com


It was too effective.


I don't get why Google doesn't give an option to permanently remove or downgrade certain domains, straight from the results page. There are results I am never interested in. It could also work as a further signal for their ranking, if used with some caution.


Probably Google would do good to give up total control and obscurity and put people in the driving seat of search customization. Maybe that would even save them from the anti-trust push (one indexing and retrieval back end + multiple ranking, filtering and UI front ends).

But I'm wondering if this would not be better implemented at browser level. Can we have the white/black list run locally in the browser, re-ranking results on web pages and hiding what we don't want to see? Not just on Google, but also on the rest of the web, like ad blocking extensions.


I have an account there and I use it but the interface is atrocious and they seem to actively be making it worse.

Recently they removed the facility to, when saving a pin, to type the name of the board to find it, now having to scroll through an alphabetical listing.

Before that, they also redesigned the website such that a refresh of the page was required in order to save a pin to two different boards, an action they explicitly support.

The website is also a slow resource hog.

Of course, it comes with the tried and tested “corporate morality” where there are rules against various things that are phrased in terms of moral concerns that are really about keeping their app on the Apple store — this naturally includes vague standards on “nudity” that are not worded in a gendered way, but there is an unofficial code that everyone knows that it very much is.

The most annoying moral preaching I encountered was that I searched for “boy's love” once — the genre of male–male tragic romantic fiction, that got me a preachy message about pædophilia that made me suspect my i.p. address was probably passed on to some U.S.A. authorities. It's really quite a big genre and has nothing to do with pædophilia.

They only recently added a way to search one's own's pins, and it's very lacking and doesn't reliably even catch words in the descriptions thereof. Sometimes it does; sometimes it doesn't.

What can be said about it is that the algorithm for finding related things is generally quite good — the interesting thing is that it's actually better at finding the aforementioned boy's love fiction than most of the websites that actually host it.

The problem is that there is no real competitor.


> I have an account there and I use it but the interface is atrocious and they seem to actively be making it worse.

If the user is confused on how to do something then it keeps them on the page longer and pushes the metrics up

taps on head


The irritating part is that Google used to penalize websites dramatically if the user couldn't get to the content but the search engine could.

I guess that interfered with Google extracting ad money.


Yes, Ointerest and Quora are types of site that Google should really punish hard. Account requirements is something that should move sites outside the few top pages...


Yeah, the results should definitely contain a HEAVY waiting for accessibility by the general user and most people don't have a pinterest account nor do they want one.


The absolute worst part of pinterest is the the time it takes to respond to ctl-w. It's slower than just about every other site.


With Google featuring progressively more ads and Google content at the top of the results page, it's not clear if it's still in Google's self interest to give users the best results anymore.


This is why there's an aggressive firewall between the ads and search organizations. The ranking team isn't even aware of revenue numbers by ACL.


I'm sure the search team can, erm,google for the ad revenue figures in the earnings release.

After reading the anti trust filing it’s hard to argue Google is anything else but evil and a lot of what it does is to hide its monopoly.


Apropos of anything else, Google is famous for a "monorepo". Any team can see the underlying architecture of things, regardless of any supposed "aggressive firewall" alluded to above.


A monorepo does not let you deduce the revenue consequences of experiments, but regardless, even if you could figure it out, it isn't part of the incentive structure.


You think the search team runs an experiment once per 3 months to test how their algorithm ideas impact earnings?


The company isn't a charity. If a search change demonstrably has a negative effect on company earnings (even if users like it) it will almost certainly be reversed. I don't think they run an experiment once every three months, I think they are testing continuously, including A/B testing (some users get the change, others don't,).


> If a search change demonstrably has a negative effect on company earnings (even if users like it) it will almost certainly be reversed.

This is completely false. I've personally been involved in launching ranking changes that negatively impacted revenue substantially. I didn't find out until months later through the grapevine, and only in vague terms like "ads had to scramble." Versions of them remain in production today. Revenue projections were never considered as part of the launch decision, nor have they they been consulted as part of evaluating subsequent versions.


Perhaps this has gone too far, as others have noted, there are so many ads and Google generated content (esp. YouTube links) that the organic search results are practically on the second page.

If Googlers were allowed to consider revenue in the search results perhaps they wouldn't have been so easily down graded to a second class citizen.


You can find public examples where the opposite has been done.

https://en.wikipedia.org/wiki/Google_Panda

> In 2016, Matt Cutts, Google's head of webspam at the time of the Panda update, commented that "with Panda, Google took a big enough revenue hit via some partners that Google actually needed to disclose Panda as a material impact on an earnings call. But I believe it was the right decision to launch Panda, both for the long-term trust of our users and for a better ecosystem for publishers."

Disclosure I work at Google.


I doubt the ad organization and ranking teams decide where and how many sponsored listings appear in search results, which I believe is GP's point: the page design/layout and the proportions of ads vs. (more or less) organic listings. Someone decides that searches have e.g. four ads at the top and two at the bottom.


I’m... skeptical about this firewall


If you believe this I would love to sell you an extremely expensive bridge.


If ever.

Well to be fair, probably the engineers want to on some level, but there is a business mandate to not (and there always has been).


Yeah, it’s honestly time for a new search experience


>Google should be asking from a product perspective if they are providing the search results the user is most likely seeking.

I am cynical. Maybe Google's own experiments show that users click more ads when they get frustrated of the Pinterest spam that is technically not spam enough to be removed from the index.

What the users are going to do? Use Bing?

Google's most optimised version is probably being an Ads search engine where you are presented only with the most lucrative ads that are relative to your query. Essentially Yellow pages.


>What the users are going to do? Use Bing?

duckduckgo isn't horrific. And certainly not for image searches, which is how Pinterest's pollution ends up being noticed.


I had an experience the other day where duckduckgo worked where google had failed (an obscure java default bug). It was pleasantly surprising.


It's certainly going in that direction. With reverse image search being replaced with "Search with Google Lense" on mobile, the most prominent results are now all Google shopping ads.


If that's the idea, isn't it easier to just show random junk? Why pinterest?


> I seriously doubt that most pinterest results are what users are seeking.

Well, yeah, since even if you click through Pinterest won't actually show you the content.

Isn't that supposed to disqualify them from appearing in search results at all?


Just do -pinterest in your search to filter


User: This service has a problem and it's not working well.

Programmer: There's nothing wrong with the service, you just have to use [magic incantation] to make it work right.

I've been hoping for 20 years that this kind of exchange would stop happening. Still hoping! :-)


And there's always this kind of response. They didn't say the service was fine, they just provided a simple workaround which doesn't require forcing a change to Google's algorithm. Hoping they don't help with this seems odd.

So, to counter, I'm glad that they did say this as it regularly annoys me too.


It's indeed a useful tip - and no offense to the GP - but this kind of reply always feels like the tech support equivalent of a mandatory arbitration clause: like an attempt to solve an issue individually with each person reporting it instead of giving a systemic solution that would benefit anyone. The blame is shifted from devs providing a good service to individual users not knowing the latest workaround.


Most probably the devs know how to do it but specs written by someone else told them to obfuscate useful functionality.


Google: we need your personal information so we can improve your experience

User: -pinterest

Google: we need your personal information so we can improve your experience

User: -pinterest!!

et cetera, ad infinitum ...


But no one said that there’s nothing wrong with the service. They merely offered a very simple workaround. I can’t imagine why you’d hope for that to stop happening.


Why? This is a quick fix. Maybe the problem will get fixed "officially" someday. But I have the problem now, I need a workaround now, not in 5 years.


User: your thing doesn't work the specific way I want. I haven't read the manual and refuse to learn now to use its features. You should make it magically work the way I imagine.

Multiply by 100 users with different opinions, all entitled, on how thing should work.


Services and users have different goals. Why is the helpful technologist the problem?


Real answer?

Because helpful technologists are the interface between less knowledgeable users and services like Google. We’re the power users and “mavens” who are both highly exposed to the product and have some understanding of what’s going wrong and how it might be made better. We also spend a lot more time talking to that product’s engineering staff than the typical user (see e.g. the fact that we’re here on HN.)

When helpful technologists opt out of the problematic aspects of a product by uttering some magic incantation, we essentially remove ourselves from having to be annoyed by the product’s rough edges. This means we’re less likely to help improve the service for everyone, and we’re probably more likely to incorporate similar carelessness into the things we make ourselves.


How could a technologist, who's not in charge of the $BigTech be able to fix the problem? They can't. So, they offer the user a work-around. Then, the user never complains to $BigCo because of the Help and still nothing gets done. And later, we complain that we're not fixing $BigCo and we're all mad at "the help".

How, really, could you improve the Google service for everyone?


How many Google Search* engineers saw this article on HN? I guarantee the number is greater than zero.

* Or people who talk to Search engineers regularly, etc.


Acausal decision theory, given the assumption that you have the same decision-making algorithm as the technologists in charge of $BigTech. (The assumption's completely invalid, so it doesn't actually work.)


I see your point. Working around abusive behaviour, and accepting that as a solution, just enables further abuse. I don't think we can influence said engineers here on HN. I know for a fact that I personally wouldn't be influenced, and I also think that if the product managers will want something, then it will get done, no matter what people write here.


Your second statement is enabling bad behavior, your first statement is insane.


No. They're exactly right. The root problem here is incentive misalignment. Both Google and Pinterest sacrifice value they provide to extract more revenue from their users.


Care to elaborate on the insane part? I feel sane.


"Programmer" here is wrong. Because we are all user of the product "google search".

But yes I see your point. I call it the "permanent workaround"


I think your premise is wrong. The service isn't broken. I'd assume Google has enough people and data to know that it's doing what most people want. They were even kind enough to put in a workaround for the minority for whom it doesn't work well (which appears to be OP).


Most people want to have a full result page of nothing but login walls when searching for any kind of image?


Most people who click on Pinterest links are already logged into Pinterest...


There’s no way that’s true. I’d be surprised if even 1% of the Google users paid attention to the source site underneath the image.

And you’d think people with a Pinterest account would opt to search on their website/app rather than going through Google (thus making it even more unlikely that the Google user has a Pinterest account).


Pinterest has a huge user base. They don't need to look at the source if they have an account; the link will work fine.

And, no, most people don't bother picking a site before searching. Why would they artificially limit their results? People don't even visit amazon before searching for Amazon products.


Pinterest’s user-base is surely minuscule compared to the amount of users using Google Image Search.

I’m not saying that 100% of all users with a Pinterest account use their website/app, but many of them will of course opt to use Pinterest’s website/app for their image searches, which means an even smaller percentage of the Google Image Search users will have a Pinterest account.


Not just Image Search - Pinterest often dominates Text Search, and people here are "Oh well, they figured out how to deliver what Google wants!"


But not most people doing a Google search. Anyone who is not on Pinterest has its result page spammed with useless links.


For the amount of people hired you would assume they had the best people.

But their hiring process is broken. They randomly hire a % of people who pass an algorithm test. It's so random that 70% of their workforce wouldn't be hired again if they had to go through that process.

The brand is so strong that people think anyone who makes it into google must be the best of the best. It's more like a random person who has studied leetcode.


Google account preferences used to have an option to exclude domains from results. About a decade (?) ago I added Burleson Consulting and expertsexchange to blocked domains and Google was instantly useful again. In 2014-ish Google moved this functionality to Chrome so I lost access to it. Not sure if it is still there.


It's not perfect, but I use the Firefox version of this extension. Good for blocking Pinterest and those techno-babble scraper sites.

https://chrome.google.com/webstore/detail/personal-blocklist...


It would be great if Google had a little "hide this domain for me permanently" next to every search result. Not only would it improve that user's own search results, it's a very strong signal that other users also might not like it, for de-ranking purposes.


They used to offer that functionality but removed it years ago.

I've been using this userscript since then. https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...

It might be called Google Hit Hider by Domain but the author has expanded it to cover most major search engines over the years.


I've used a similar script; the problem is, for certain searches, literally every result is a pinterest link, and you end up with a blank page.


I just wanted to mention that excluding strings has been broken for me off-and-on over the last month at least.

Not directly relevant to anything except how lame google has become.


Only a month? I don't recall the last time it actually worked. I always end up having to do -"string" and put it in quotation marks, kind of like telling Google "why, yes, this thing I put that minus sign in front of really is something I don't want to see in my results."


Clearly we should not have no include -fakenews -porn, etc, otherwise I might as well use bing / Duckduckgo/ whatever


-site:pinterest.* (there's supposed to be an asterisk at the end, but I have to type something after the asterisk for it to show up)


It gets tiring.


Thank you, but still, the regular person is not finding what they seek, but what SEO experts want them to find.


Or just avoid using google. The Pinterest results in google image search queries was a major driver to me using bing/ duck duck go.


Linking directly to the image file (and in fact a cached version of the full image) is one of the best reasons to use DDG.

Reverse image search is also intentionally gimped, unlike non-US jurisdiction Yandex which is quite good.

It's certainly for legal reasons, not technical, but Image Search has gotten progressively worse every year. It used to be godly effective 10 years ago.


> Linking directly to the image file (and in fact a cached version of the full image) is one of the best reasons to use DDG.

That’s possible thanks to Microsoft’s Bing API. I’m surprised Getty (who’s afaik responsible for Google removing direct image links) only went after Google and not Microsoft.


At least for image search, I use a Firefox extension called Unpinterested. I'm very happy with my search results now. Note: I had to toggle an option to make it work only for image search, because it borked other Google results.

I remember when Google added a functionality to block domains entirely from your results. They removed it soon after though; I'm guessing some big advertiser was blacklisted many times.


Both in Google, and in DDG, sometimes when I am looking for an image (i.e. image of a "full body workout plan") the images I find tend to be 'hosted' in Pinterest, but in reality following them links take me to Pinterest, and then back to the original source (which is some random fitness site).

I find Pinterest useful as an image search engine.


"That means I spend less time using google, and that in theory means less money for google."

In practice, losing customers means less money for Google. Pinterest is a large Google customer. As individual users, we are very tiny bits of a very large target, but we are not customers.


> Google should be asking from a product perspective if they are providing the search results the user is most likely seeking.

I never want Pinterest. I don't have an account. I don't want one. I will never want one.


It is basically an arms race between search engines (specifically google) and SEO. Google does change their algorithms, but with enough effort (and money) SEO proffesionals can figure out how to tweak their sites to keep gaming the system. And the casualties of this war are the authentic sites that don't spend a ton of money on SEO.


This is a little bit like: 'McDonald's should serve healthier food, like Salads'.

'What Google Wants' and 'What We Want' have reached a crude equilibrium and it's unlikely to change that much, even if there are obvious improvements to make.

Qualitative improvements beyond need generally require a very specific corporate and cultural focus. If Sergei or Larry 'came back' and mandated it, the system would move around it. Or Sundar could, maybe, pull that off, but it's not in his DNA really.

The lack of material innovation in search stopped over a decade ago.

Otherwise, we should be getting betas every few months, with all sorts of options, features, etc. Even simple things like 'forever blocking a site by default' would be great.


They are playing a cat and mouse game, only the mouse is quite determined as its bonuses depend on evading, and the cat is very fat and comfortable :).


Stop using Google search. DDG works just as well


I use DDG search most of the time, but I find this to be untrue. DDG is pretty good, but struggles with context a lot more than Google does. For example, when searching for an error in a programming library, DDG is likely to return the library homepage, whereas google is more likely to return the specific github issue where the error is being discussed.


For text it depends [1] but for images I find DDG to range from slightly to significantly better than Google. Also, I have direct access to the image file URL without having to rely on browser extensions.

[1] I wonder if the meme "DDG is not that good" is so prevalent on Hacker News because it sucks for programming stuff.


Yes, Google image searches produce wild results. Far-right groups have had a field day with it.


This used to be true. Regularly I had to use the !g bang to find what I was looking for. Lately, I've realised that I hardly use it anymore and I do results are far worse on Google. Your mileage may vary of course but I don't bother with Google anymore.


Same. I went the DDG route a couple of years ago, and searching tech topics made me constantly do g!. And a few other things. Suddenly, Google's results go so much worse, that the duckduckgo offering just proved to be actually functional by comparison. And I think it actually has improved over the years. It's now my default browser search and I'm comfortable not having to worry about Google capturing every feeling or thought I have when I google something, and storing that for eternity.

Not to mention the awful vendor blogs and stores and secretive marketing clickbait masquerading as a neutral information totally dominating Google's page 1 and 2 of results.


Agreed. I switched to DDG a few weeks ago as a default. I appreciate the privacy, but for some categories (like tech questions) the results are just... not as accurate. For general queries they're comparable. For searching specific issues, Google is more likely to show relevant discussion threads of the exact issue.


Would that this were true, but it's not. Searching for specific technical issues especially shows DDG's weakness.


Outside of searching for specific technical issues, DDG is pretty good. I use DDG for most of my searches and this is the first I've heard of search results being dominated by Pinterest, so it's at least better on that front.


That's problem, DDG works nearly exactly the same (except a bit worse imo). DDG channels a lot of Bing, which virtually clones Google which makes it not much of an alternative at all.


I’ve been using DDG as my default for years and it really fails to provide me helpful results for programming and Linux questions. I know this because I frustratingly have to add !g to queries all the time lately and I’m questioning my choice to default to DDG.

That said, I don’t like being tracked. It’s not clear to me if the quality of results I am seeing are due to Google’s tracking or not. (I suppose I could research this but I’ve not done so.)


It has improved for me to the point I don't use !g anymore. Maybe subconciously I now do queries that fit DDG better. Anyway, I get the answer I want so can now live without Google.


Instead of playing cat and mouse with Pinterest, what would be a much better idea is to let users themselves decide what's important to them. Let us vote search results up or down, and let us block sites we don't like yet keep showing up. Those two changes would do a lot to improve user experience.


I think Google collects clickthorughs on the search result page and thus knows how people react to its responses. It also knows when the user is quickly returning to search after checking out a link. Pinterest is probably appreciated by people, probably of a different demographic than you.


Here is some background on this:

Recently, it has been revealed that Pinterest reverse-image searches Google for uploaded images and adds the Google query result to the Pinterest image page for SEO: https://www.rankscience.com/blog/pinterest-image-seo-growth-...

This degrades quality of Google image search, for example this Reddit post on removing Pinterest got 77k upvotes:

Reddit r/LifeProTips to remove Pinterest from Google searches: https://www.reddit.com/r/LifeProTips/comments/hejbl0/lpt_whe...

Earlier Reddit petition to Google to remove Pinterest from search results (66k upvotes): https://www.reddit.com/r/google/comments/85atho/pinterest_ne...

Hacker News discussion of Pinterest's Google scraping: https://news.ycombinator.com/item?id=23701998

Earlier discussion on HackerNews (1.1k upvotes): https://news.ycombinator.com/item?id=16613996

"Unpinterested" Chrome extension to combat Pinterest spam (10k+ downloads): https://chrome.google.com/webstore/detail/unpinterested/gefa...


I agree, but if you take uninformed users (or users at random, that don't subscribe to a Google related forum) they would be happy with the Pinterest results because the results look nice at first sight and they don't have time to learn about the problem. Google using this signal would conclude that Pinterest is OK for the users.


I think they can't do anything about it. Regulators and lawmakers in the US and EU would be very receptive to complaints from pinterest if google were to act somehow.


Do you have references for this? I'm genuinely curious if this kind of thing is actually being done. Google's altered it's algorithm to remove low quality sites in the past and if anti-trust worries are stopping that now, well, I can't even.


If google were to downrank pinterest, it would seem very similar to the foundem case: https://www.ft.com/content/88f26f10-5d3a-11e9-939a-341f5ada9...


Google has shown time and time again they do not worry about providing the search results the user is most likely seeking. They monitor ad revenue and adjust accordingly.


Still odd that Google tolerates this. Isn't this exactly the kind of black-hat SEO that they are usually downranking or banning sites for?

Pinterest landing pages are intentionally unusable for anyone without an account. It's hard to make a good-faith argument that those pages are the "most relevant result" for someone making a simple image search.


Yep, exactly. I find that Pinterest is slowly making me less and less likely to use Google. Pinterest shows up as the dominating search result even or very specific long tail keywords where one could argue that a collection of images on a login-gated page is not helpful at all.


I don't use google for images anymore because of this.

Bing is terrible, but it's better than google.

Which means that google is worse than terrible.


Yandex is great for images these days.


> Isn't this exactly the kind of black-hat SEO that they are usually downranking or banning sites for?

How is it blackhat to do exactly what Google wants?


Using tricks at the detriment of the user to the benefit of the service is blackhat. Pinterest is doing these tricks to their benefit at the extreme detriment to everyone but themselves.


It's weird to blame Pinterest for Google's choice to promote the site. Sure Pinterest wants to rank highly, but they aren't tricking Google. Google knows how popular they choose to make Pinterest results, and how much most users like those results.


If I make an algorithm to rank sites, and you figure out the internals of that algo and manipulate your behavior to rank at the top, that's bad and not the same as being Google's choice, at least not directly.


My friend, Google tells you how this works. Pinterest doesn't have secret knowledge.


"There's no intentional or manual effort at Google to promote pinterest."

But there is an effort at Pinterest to promote itself via Google. Pinterest pays Google for online ad aservices. Pinterest is a Google customer. The user is not.

The bottom line at Google is not driven by showing users the "best" results for users. The bottom line is driven by customers such as Pinterest who pay Google to assist them with online advertising.

Page and Brin identified this conflict of interest in their 1998 paper introducing Google, which was originally intended to be an academic endeavor, not to become an online ad services company.^1 According to the authors, (a) the interests of users and advertisers are not one in the same, (b) there are inherent effects on search results if the search engine is funded by advertising, and (c) the behind-the-scenes operation ("technical details") of search engines funded by advertising become largely non-transparent to users/competitors (a "black art").

I reckon (c) is why we get questions like this one from the OP.

I have also seen these inexplicable and useless Pinterest links in Google searches.

1. https://research.google/pubs/pub334.pdf

Quotes:

"To make matters worse, some advertisers attempt to gain people's attention by taking measures meant to mislead automated search engines."

"Up until now most search engine development has gone on at companies with little publication of technical details. This causes search engine technology to remain largely a black art and to be advertising oriented (see Appendix A). With Google, we have a strong goal to push more development and understanding into the academic realm."

"Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users."

"For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

"Furthermore, advertising income often provides an incentive to provide poor quality search results."

"But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm."


The term "black art" appears to be a synonym for "black magic".

Wikipedia (https://en.wikipedia.org/wiki/Black_magic) describes black magic as "the use of supernatural powers or magic for evil and selfish purposes."^1

1. J. Gordon Melton, ed. (2001). "Black Magic". Encyclopedia of Occultism & Parapsychology. Vol 1: A-L (Fifth ed.). Gale Research Inc. ISBN 0-8103-9488-X.


Had anyone ever presented evidence that advertisers manipulate organic results? Former employees blow all sorts of whistles. Why has this one never blown?


pinterest is clearly being spared seo penalty so one can only assume pro-pinterest bias at google


What they are doing to get into google search is not the problem, the problem is all their links are scams, clicking on pinterest google images link lands you on a login screen, or on a farm of similar/adjacent images. Nowhere do they have the content you were actually looking for.

TLDR: they are gaming google


It is there— it usually just takes pages of scrolling. I heard a 2nd-hand account which said their product team leadership knows this annoys the hell out of people, know it's an easy fix, but they do it anyway. If that's true, I'd guess they're ignoring Goodhart's Law with their engagement metrics and need to re-focus on their UX, though I don't have any first-hand insight into it.


Not if you don't have Pininterest account. then you can't scroll more than half a page.


One could say they are gaming it


The Wikipedia clones gamed it and they were deranked. Why does Pinterest get special treatment?


I'm guessing because Pinterest buys ads from Google? They're the customers from Google's PoV.


Related: in the last several years, there started appearing these StackOverflow mirrors. It's gotten to the point that you can't google a programming question without running into at least one. Some copy the questions and answers verbatim, some use machine translation to (crappily) translate them into my native language. Sometimes they even rank higher than the real StackOverflow.

I really wish there was a non-hacky way to ban sites from Google search results. I also feel like Google's ranking algorithm is utterly broken since it's amenable to this kind of exploitation.


I've seen the same, but with GitHub scrapers. One is called something like HappyGit, another is called GitMemory. And a lot of times Google thinks GitMemory is the original, so the _actual_ original source (GitHub) never even appears. So then I have to go to GitHub and try to find the original myself using snippets of text. Really makes my blood boil.


I've even had one of those spammy websites appear as the first result, in a "featured" box. It's just sad. https://news.ycombinator.com/item?id=25437378


They're pretty old. I remember when Google added an option to block domains (the option you want) around 2011 and they were the first thing I removed from my results, along with Wikipedia mirrors.

After the option to block domains was removed around 2014-2015, I rarely came across those anymore, but it seems they're back...


There are extensions to permanently block sites from results.

Like: personal blocklist https://chrome.google.com/webstore/detail/personal-blocklist...


I know these exist and I could as well write uBO rules to hide unwanted search results, that's why I said non-hacky ;)


Add this to your browser's search engines:

  Stack Overflow
  so
  https://www.google.com/search?q=%s (site:stackoverflow.com OR site:superuser.com OR site:stackexchange.com)
Then just begin the search by typing: so[spacebar]<your search term>


I find those are actually useful from the perspective of being mirrors of content that the real SO has deleted for whatever reason.


I presumed it was just DuckDuckGo lacking algorithms to detect such 'spammers'.

But now I learned that Google is suffering the same poor detection. Another 'downside of using DDG debunked', I guess.


We are so biased towards google search being the perfect technology, that every flaw a competitor has is simply ignored on google.com


I've been seeing the same even in English language.


One big reason why Pinterest ranks so highly, and why you are seeing different tld's in the results, is because they have a variety of tld's that they use and those tld's all have separate crawl budgets from each other. So Google crawls Pinterest about an order of magnitude more than most other sites. You can read more about this here: https://medium.com/pinterest-engineering/how-switching-our-d...

Source: I run a think tank focused on Google’s web crawling advantage and have been studying stuff like this for a couple years now.


Does your think tank come up with any ideas on how to keep companies like Pinterest from peeing in the pool so to speak? They made it impossible to find DIY craft stuff that I used to search for, and I can’t imagine it’s the only site doing it.


We are pursuing ideas that we think will increase competition in the search engine market. Our main focus right now is making Google’s index of the web available for use by competitors, since no one other than Google is really allowed to crawl the web. If people were allowed to use the index to build what they want, then you or somebody else would be able to build a search engine that avoids the Pinterest cruft and would be more suited to your needs, instead of Google’s one size fits all approach. So, to answer your question directly, yes we do have some ideas about how to do this.


> making Google’s index of the web available for use by competitors

That does seem like a good idea since it amounts essentially to a database of stuff that Google does not own (by construction) and is of public utility.


Exactly. Everybody always talks about turning tech companies into public utilities without really explaining what the utility would be. A public index of the web would be an amazing utility for many of the reasons in this thread and would spur on a ton of new innovation and businesses.

If you would like to read more about all this, please checkout https://knuckleheads.club


The pursuit of a public index is excellent.

Some feedback:

- A cost of membership runs contrary to establishing this group, especially at such a high recurring charge.

- I'm not sure what your software/AWS situation looks like, but 20 million robots.txt files acquired from Common Crawl is something I can analyze on my PC. It doesn't seem to presently justify such high costs.

- Prioritize building a mockup index with an intuitive frontend. This is essential for non-technical people to understand

- Exclusively talk with EU legislators (they are motivated, whereas nothing will happen in the US).


Thank you for the feedback!

I think the price for membership dues is reasonable and many people agree evidenced by them signing up. I think I might start a petition that is free to sign up on though, thank you for the inspiration!

It is possible to analyze those files on the pc, it just takes a much longer time. The analysis is an iterative process and so the faster the computers the faster the iterations and process go. I was analyzing them on my pc with python for the first year until it got too slow and my I am using an aws server with some rust and that is going much better. I also need to increase the number of files analyzed by about two orders of magnitude soon as well.

Great idea, very cool. That’s going on the todo list!

And I am going to be reaching out to and speaking with whoever is interested. One of the fun things about this is that it is an international dynamic, with some jurisdictions having abilities that others don’t. For example, the UK CMA has subpoena powers that the US Congress lacks and got a ton of information out of Google and Bing that shocked me. The US has the ability to get the CEO’s to show up to hearings while the UK does not in the same way. Why limit ourselves to one government when there are so many to mix and match from here?


What kind of research is it? Is it just funding you or members can participate? I do Bayesian and non Bayesian data analysis and modeling, would be potentially fun to poke at the data


Right now the research is two parts, analyzing websites robots.txt files for bias and then talking with search engine operators and website operators to get their stories that validate these ideas. Right now I am the main person working on this research but I would like to get the robots.txt parsing and analysis code open sourced soon to allow people to start digging in. Getting people access to the data is trickier, but it feels doable as well. You'd be welcome to join in!


> since no one other than Google is really allowed to crawl the web.

??


There are two main reasons why I say nobody besides Google is really allowed to crawl the web.

The first is that Google gets much more access to pages on websites than everybody else. You can see this by examining the robots.txt files of various websites[0]. I've been doing this for several years now and Google has a consistent advantage across many thousands websites that I've looked at. This adds up to a significant advatnage and many search engine operators complain about how it hampers their ability to compete with Google[1].

The second is that Google gets to ignore crawl delay directive in robots.txt while other search engines don't[2]. Website operators cannot tell Google how fast they want their website crawled, they can only request that Google slow down. If another search engine tried to do what Google does, they would likely be blocked by many important websites.

If you would like to read more about this, please checkout https://knuckleheads.club/

[0] https://pdf.sciencedirectassets.com/robots.txt

[1] https://www.nytimes.com/2020/12/14/technology/how-google-dom...

[2] https://www.seroundtable.com/google-noindex-in-robots-txt-de...


So, uh, don't respect robots.txt in your search engine? It's not like there's a law that you have to, and that you can't pretend you are Googlebot. The only real obstacle I can imagine is that some firewalls might be configured to be more permissive with traffic originating from Google subnets.


You would be blocked fairly quickly by many website operators and no longer able to access those websites if you straight up ignored robots.txt files. You also might even end up being served cease and desists by some websites and sued if you continue to persist and try to find ways around it.


And what if you do respect it but follow Googlebot rules?


Applebot was able to get away with doing exactly this but I imagine that's because it's Apple and websites knew that Apple was about to send them enough traffic via Apple News to make it worth their while. I don't know if other search engine operators have tried this but I would imagine they would get caught by rate limiters set for non Google IP's and then they would be blocked.


Still, you keep saying all that as if most websites even notice that they're being crawled, and that their operators are very aware exactly when by whom they're crawled. Like as if the admin gets a notification every time a crawler comes by or something, with precise details about it. I don't think it's nearly as serious as you're trying to make it look.


I've been a part of a team that operated a large website and I've been paged before because of the issues that somebody was causing because it was being crawled too much. Many people in the web operations field have had the same experience. Generally speaking, the larger the website, the more sensitive they are about who is crawling and why.


To add another data point for you: I have had one of my websites brought down by Yandex bots before. There are also dozens of no-name bots (often SEO tools like ahrefs, semrush, etc.) that can sometimes cause troubles.

For me it was a problem of having lots of pages, and having a high cost per request (due to the type of website it was).

For other websites, it is not necessarily about the volume of traffic from bots, but the risk of web scrapers getting their proprietary data. They're fine with Google scraping their info because that's where their traffic comes from. They're not okay with some random bot scraping them because it could be taking their content and republishing it, or scraping user profile data, or using it for some nefarious/competitive purpose.


> the risk of web scrapers getting their proprietary data

That's some weird logic, to me at least. That data is literally given away to everyone but some people or organizations can't have it? If you want to control access to it, maybe at least require people to register before they can see it? Is it even proprietary if it's public with no access control whatsoever?

This for-profit internet is just really such a parallel universe to me.


This is a question the courts are working through with LinkedIn and HiQ https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l... as well as the Van Buren case https://themarkup.org/news/2020/12/03/why-web-scraping-is-vi...

It’s a different world where there are no laws or prices or contracts really.


> This for-profit internet is just really such a parallel universe to me.

I know I have been a contrary commentor in this thread, but I hear you with this. What a monster we have built, and what always gets me is how trivial everything is. So much capital is flowing through these ephemeral software systems that, if gone tomorrow, would be ultimately inconsequential to mankind.


I mean it's ridiculous to think about it, but there's this giant, many-billion-dollar online marketing industry that I essentially don't exist for. If it's gone tomorrow, I would indeed not notice, but it'd be the end of the world for some.

> and what always gets me is how trivial everything is

Whenever I read about corporations and how they work, I always inevitably ask myself the question "where the hell does enough work to keep this many people busy even come from". Everything is ridiculously overengineered to meet imaginary deadlines.


> That data is literally given away to everyone but some people or organizations can't have it?

It's often a question of quantity. LinkedIn probably doesn't care about you scraping a few profiles, but if you're harvesting every bit of their publicly-available data, then they get a little scared that you're building something that's going to compete with them.

Same with Instagram, or Facebook, for example. Though in this case it's probably more of a user-privacy issue - at least that's what they say.

It's not really weird logic to me - seems to make sense.

> If you want to control access to it, maybe at least require people to register

Most of the time they can't do this because they need the Google traffic. LinkedIn wants a result in the SERP for Bob Smith when you search for "Bob Smith" because that helps them get signups. Google won't list the page if that content is gated by a sign-in/register page.


There are syndicated blacklists that get fed into automatic traffic filters. Not to mention a surprising amount of the web is fronted by Cloudflare and other CDNs, making that kind of traffic detection and blocking more effective and widespread than you might expect.


Google tells website operators how to verify google bots in a way that can't be spoofed.


It's a situation where the rules seem obvious but the practical realities of it mean Google has the advantage by being the incumbent. No one would dare block Google for a search traffic reliant business, but some upstart search engine will quickly end up on blacklists even with reasonably slow crawling.


> Our main focus right now is making Google’s index of the web available for use by competitors

How would that work? I'm pretty curious about this if there's anything out there to read.

EDIT: Ah I saw the link to Knuckleheads' Club :P


:) No worries. If you'd like to chat about it more, shoot me an email at zack@knuckleheads.club, I'm obviously very passionate about the subject and love talking about it. There's still a lot more to write and publish, I won't be done with this hobby horse of mine for a while yet.


Wow, such index would tremendously help free p2p search YaCy, https://yacy.net.


> making Google’s index of the web available for use by competitors

What incentive would Google have to continue populating that index?

Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it?

> since no one other than Google is really allowed to crawl the web

Maybe this is the problem that needs solving.


> What incentive would Google have to continue populating that index?

Presumably they would still want to run google.com and make money off of it.

> Would I be breaking the law if I independently crawled and hosted an index without publishing an API for it?

No. You would not get the advantage that Google gets when it crawls the web and so would not have access to a large amount of data that nobody else has access to.

Updated based on edit of parent post:

> Maybe this is the problem that needs solving.

Why have websites waste the money to serve all those requests all over again? Why don't we have Google share the results and we can use that money to do more productive things than recreating that work? I don't think website operators would be happy if there were a hundred more crawlers out there crawling as much as Google does now.


Do any site operators actually block non-Google search engine crawlers because being listed DDG/Bing/etc isn't worth the extra cost of serving the crawler? It sound a bit ridiculous unless they actually don't want to be found. Maybe they only allow GoogleBot because that's all they thought of and the extra cost is in researching what all the other search engines call theirs.

Perhaps other search engines should spoof GoogleBot. Browsers have being doing that since forever spoofing Netscape (Mozilla), Safari, etc. for the same reason.

> Why don't we have Google share the results and we can use that money to do more productive things than recreating that work?

This sounds like a common fallacy of people criticizing the free market. Duplicated effort looks wasteful but turns out to be far more productive than the lack of incentive that comes with not being able to profit from your work/investment.


> Do any site operators actually block non-Google search engine crawlers because being listed DDG/Bing/etc isn't worth the extra cost of serving the crawler?

Many website operators do actually block crawlers from non Google search engines and it's because the cost of being crawled isn't worth it to them. Here's a good quote from one such webmaster:

    As a webmaster I get a bit tired of constantly having to deal with the startup crawler du jour.

    From law firms looking for DMCA violations to verticals search engines, to image aggregators, to company intelligence resellers… It feels to me that everybody and their brother has gotten into spidering sites.

    With 10,000s of pages that have content that is only relevant to a targeted audience who is perfectly able to find us on the majors, I do not hesitate to block (and possibly ban) when I see an aggressive crawler that does not provide me or my customers with direct benefits.
Taken from http://www.skrenta.com/2008/04/cuill_is_banned_on_10000_site...

> Perhaps other search engines should spoof GoogleBot. Browsers have being doing that since forever spoofing Netscape (Mozilla), Safari, etc. for the same reason.

People have tried this and it doesn't work. Google provides ways to check to make sure traffic is coming from Google IP addresses and practitioners and academics study how to spot fake Googlebots. https://developers.google.com/search/docs/advanced/crawling/... https://blogs.akamai.com/2014/07/search-engine-impersonation... https://ieeexplore.ieee.org/document/8421894

> This sounds like a common fallacy of people criticizing the free market.

I am asserting that crawling the web is a natural monopoly. This means that the free market has failed and that it is not possible for the market to heal itself in this regard. There is significant evidence that this is the case and I imagine you'll be hearing more and more about it soon.


I would think the site owner’s cost of being indexed is the same for every search engine that indexes the site.

The benefit varies with the quality of the search engines, and that will vary between search engines, but it does get larger the more a search engine is used, so a cost/benefits analysis may show Google and a few other large ones are the only ones worth supporting.


Yes! Exactly!


Yes, site operators actually block non-Googlebot crawlers. See the example [0] from https://news.ycombinator.com/item?id=25538842 .

Spoofing crawler identity completely defeats the point of the honor-system robots.txt.


This I did not know about and I'm surprised they don't get penalised by Google in some way at the display of search results stage. I mean sure, I suppose one can do the -pinterest trick, but it's still a bit weird that Google thinks search results that are pretty much the same site but from different tlds are not spammy. Do you think they will change this to prevent Pinterest from essentially spamming some search queries by using the tld hack?


Live by the SERP updates, die by the SERP updates. It's been at least two years now and Google seems fine with it (until maybe one day they aren't). I know it might annoy folks like you and I to get reposted pinterest images in our results but it might be that Google has done research and found that people like seeing pinterest in images and so that is why they are keeping them around. It's hard to say why they are there until Google says something about it, Google rankings are a black box at times.


I think others here have alluded this but the thing is that once, doing everything you need to do to get listed but actually being crap was enough to get you de-listed. The criteria wasn't "do what Google wants", the criteria was "be good and also help the algorithm determine that you're good".

Only running the right SEO wasn't sufficient then and I certain wish that criteria remained.


Who determines what is the best ranking? Who was defining what was "good"? There isn't some abstract, mathematically verifiable best ranking, a platonic ideal that all search engines should aspire to. Google has always made choices about how to rank that trade off the interests of various groups of users, from the very beginning. The complaints that you and I have about pinterest being ranked so highly come because Google has decided that our desires for how the results should look are not important enough compared to how other, perhaps larger, groups of users want them to look. Maybe Google deranks pinterest tomorrow, but they've made the choice to keep it as such for a couple years now. Whether it's because they think they have better ways to spend their time or because they like it the way it is, it's not really possible for us to know without somebody leaking that information. But, I imagine, there would be a group of people who would be as disappointed as we are elated if Google were to change this.


There isn't some abstract, mathematically verifiable best ranking, a platonic ideal that all search engines should aspire to.

Of course, "quality" applied is a broad term indeed. But there no being a single certain measure of quality doesn't mean there aren't some things that most people would call crap. Google employs a large number of search quality raters and these folks can likely distinguish the terrible from the OK.

Google is far from ideal and getting worse but you seem to be exposing a pure democracy of algorithm conformance, which is guaranteed to result in hot, steaming idiocy.


I think what I am trying to espouse is two things. The first is that there might be a large group of users unknown to you and I who enjoy looking at pinterest in the image results and that is who Google wants to cater to. And the second is not that Google is or should be a "pure democracy of algorithmic conformance" but that Google has, does and always will make choices about what is best and that the definition of best is up to their discretion. They are making choices about who to cater to with their results and justify those choices with their own logic that we will likely never know. One company having the power to be the only one making all those choices isn't anything I would call a democracy.


The first is that there might be a large group of users unknown to you and I who enjoy looking at pinterest in the image results and that is who Google wants to cater to.

The discussion is about Pininterest in text searches. My only objection to Pinterest in image searches is that it's paywalled/login-walled.

But larger issue is I'd acknowledge some search results just aren't what I in particular want but I'd claim other results are actually objectively low quality and you seem to want to push things to realm of pure subjectivity, any old crap is something someone might want and who I am to deny to them that pnis enlargement pill.

I would say that the results Google returns involve a number of factor/filters. A. What's considered mainstream, what appeals to many B. What's could be more or less objectively called quality. C. SEO, What results just slip through based on the page spending a lot of time and money appearing like A or B to the algorithm (but not being that).

And Google spends a huge amount of time and money trying to keep C from being the only thing BUT that's still not enough because there's a lot of time and money spent on the other end. At the time, this tug-of-war serves as a moat keeping competitors out. Any Google competitor would have to invest similar amounts of money.

Google has, does and always will make choices about what is best and that the definition of best is up to their discretion.*

I'd love to have a decent Google competitor. But I don't think you have won many friend here by implying that Google's discretion is just arbitrary (as a number of your posts here seem to imply to me). Google won, back when they had competition, by caring a lot about the, uh, quality of their search results. Now that they've won, they're slipping into other things and moreover, the SEO trash are nipping their heels.

Which is to say you won't build Google alternatives on "democracy" but on some concept of quality people will want.


Many people have been upvoting my comments here so the evidence I am seeing suggests that I haven't lost too many friends here with this line of thinking.

Continuing out the logic of your suggestion here, you would seem to imply that every other currently existing search engine besides Google also has bad quality as well in this regard. I say this because if other search engines had better quality that Google, they would be doing better than Google as per your suggestion. But, obviously Google is on top in a big way, so I have to ask, do you think the entire search engine industry, not just Google, has poor quality search results? What do you know that they don't?


Knucklehead: But, obviously Google is on top in a big way, so I have to ask, do you think the entire search engine industry, not just Google, has poor quality search results? What do you know that they don't?

Me earlier: larger issue is I'd acknowledge some search results just aren't what I in particular want but I'd claim other results are actually objectively low quality

It seems like you just a kind of "shtick". Anyone talking about quality, no how much nuance they add to it, gets thrown the same "how do you your ideas of quality are right". I already mentioned that quality is piece of Google formula and Google spend real money attaining that, employs thousands of people to rate search quality. Of course I'm not claiming to be a personal expert on quality. You can read further what I actual wrote above.


I don’t think making Google’s index public would be sufficient. Their real core asset is not the index but the historical engagement data they have of searchers’ interactions with the index (both on the results page and offsite analytics Google has access to, eg Google Analytics/display ad network).

Making that historical engagement data public is not feasible/realistic, IMO.


I think it is necessary though. Other search engine operators have cited this as one of the reasons their search engines fail or struggle to provide as good results. As far as the engagement data goes, it’s something that the Competition and Markets Authority in the UK has looked at as part of their investigation into google. See here for a very interesting analysis of Google’s and Bings click and query data: https://assets.publishing.service.gov.uk/media/5efb1db6e90e0...


Thanks--I learned some valuable vocabulary today! "Click-and-query data" was the phrase I did not have on hand.

I have never worked for a search engine company (or devoted much time to SEO), but I would suggest the click-and-query data advantage is invaluable to cementing Google's supremacy over any newcomers.


It was new to me as well, I think it was a phrase that was invented to describe the data during the CMA investigation but I am not sure.

I’ve heard search engine operators complain about the index consistently while they seem spilt about the click and query data. For example, DuckDuckGo and StartPage have made pretty good businesses out of pointedly not collecting click and query data. For my part, I think it’s the dual lock on distribution via exclusive agreements on being the default search engine and the advantages it has when web crawling that cements Google’s dominance over the search engine market.


Out of curiosity, what exactly is a "think tank"? Is it the sort of thing a person can make money by founding? What makes it distinct in type from other businesses or organizations?

I've only ever met one person that had anything to do with a think tank at a party about a decade ago, and I never got her contact details so I'm still very much in the dark.


Good question! A think tank is an organization that researches and sometimes advocates for the government to do certain things. They can be a non profit or an llc (knuckleheads’ club is an llc right now because it was the easiest to set up). In theory, you can make money from founding a think tank, but in practice it’s not a lucrative endeavour unless it becomes big enough that the president start earning steady six figures (like at the major ones in dc).


Some relevant prior discussions:

https://news.ycombinator.com/item?id=21622322 (Nov 2019) "Tell HN: Google should drop Quora from search results" 1000+ upvotes

https://news.ycombinator.com/item?id=16613996 (Mar 2018) “Pinterest needs to be removed from Google IMO” 1100+ upvotes

https://news.ycombinator.com/item?id=16388833 (Feb 2018)

And many more: https://www.google.com/search?q=site%3Anews.ycombinator.com+...


  google.*##.g:has(a[href*=".pinterest."])

  google.*##a[href*=".pinterest."]:nth-ancestor(1)
Add to uBlock Origin. in the "My Filters" tab. This will completely block pinterest


I miss the days when google had an option to block entire domains. So sad that they removed it.


    -indomain:pinterest


As a configuration, not per search.


I used to have a .txt file full of - options for google back in 2009 or so.

Then Google started ignoring more and more of what I searched for and they also started blocking the worst offenders so I forgot about it.

These days I mostly use DuckDuckGo for my programming searches.


Same here. Although do they have a way to block certain sites?


Unfortunately this did not work for me. I added these two, and then I did a search on images.google.com for "custom carved door frame".

5 of 7 images on the first row of results was some pinterest site.


Can you try this?

  google.*##.g:has(a[href*=".pinterest."])

  google.*##a[href*=".pinterest."]:nth-ancestor(1)


Thank you so much! A very quick copy paste into uBlock and it works wonders for Google & Google Image searches.

I wish I had looked into this before! :-)


Thank you that works, eliminated Pinterest results.


Many thanks, this does work. I do see a momentary flash of pinterest results, but they quickly disappear (and subsequent matches are re-tiled up). Good enough!


Yeah. uBlock Origin is just hiding the results, like it hides blank spaces after it blocks ads. You might notice a momentary flash only for the first few times, or on slower machines.


nth-ancestor(n) and upward(arg) look handy. They're uBO extensions.

https://github.com/gorhill/uBlock/wiki/Procedural-cosmetic-f...


ffpip is the real Santa


Weird, I don't do anything at all to block pinterest, and I just did that same search on Google Image search, and pinterest results are 6 out of 20 on the first 3 rows, and 1 per row at most for all of the rows below that.


This userscript is very useful for hiding pinterest search results: https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...

I also use it for w3schools and a few other common search result spam sites.


extension for firefox that puts "block thisdomain" under thre results: https://github.com/wildskyf/personal-blocklist


Pinterest spends a large percentage of their budget on gaming the search page.

Though I’m sure you’ve noticed in recent years that Google has begun to replace many search results with answers directly from them instead of redirecting you to places like Pinterest. For the better I say. Companies like Pinterest are parasitic and degrade the overall ecosystem of search.


Yeah - I've just been reading some of the Pinterest engineering blogs linked above about their sophisticated SEO experimentation framework and ugh, I think it's time to load up my browser with extensions or move away from google completely.

Image search has been borked due to Pinterest results for a few years now, but now my text queries have also started being dominated more and more often by their search results. I'm bummed that Google doesn't count the different Pinterest country domains as duplicates and remove them. instead there is like pages upon pages of the same Pinterest result.


> Google has begun to replace many search results with answers directly from them

> Companies like Pinterest are parasitic

Surely you mean companies like Google are parasitic?


Pinterest doesn't provide value with their image search spam hence they are parasites indeed.


I see my sarcasm didn't go over well. Replacing search results with quips from websites is parasitic as fuck.


I heard there was some incestual stuff at medium-high levels of relevant google org/teams and pinterest in terms of people moving between the companies and being friends/favors. Also that these specific teams at google are mediocre at best now and all the good engineers and product focused people have moved on to greener pastures. Probably why google image search has stagnated hard (I find bing and yandex out perform it easily depending on your searches).

The explanation that Pinterest is just "good at SEO" never made sense to me because at their scale someone at google obviously would notice and could decide to counter their SEO tricks etc (google has never had problem with doing this before).

Not to sound conspiratorial but from what I know I find it more plasubile that mid level decision makers at google can block any change that would nuke pinterest because they or their social circle have personal interest in keeping pinterest up while issue is not close to big enough or impacting larger revenue to attract attention from above.


Pinterest CEO is an ex Google worker and many investments into Pinterest come from Google related people.

They obviously turn the blind eye, anyone in the business knows that since Google is very keen on blackholing people or companies who use tricks like that. Just another Silicon Valley affluent people love story.


This answer has never made sense to me. Pinterest breaks the 2 signals I would assume Google search to use:

1. Often, the image I'm looking for isn't the main image on the page, just one of the small thumbnails and very hard to find on the page

2. Almost every single time I click on a pinterest result, I end up going back because it doesn't help me, i assume others do the same.


It should violate the duplicate content guideline, since 100% of Pinterest content is not original and lifted from another website.

Then again G also ranks those Markov-generated "blogs" with nonsensical text, not sure what to call them but you've seen them. It seems SEO is still alive and well despite all the claims of its death.


Certainly worth mentioning the WP article from October that highlights Google search's deterioration only in the last few years. https://www.washingtonpost.com/technology/2020/10/19/google-... "How does Google’s monopoly hurt you? Try these searches. Right under our noses, the Internet’s most-used website has been getting worse"

Yeah, sure, Pinterest knows how to game the results. But the rules of the "game" are in Google's control, so... I hope we don't normalize a deterioriation of a valuable resource under some neutral-sounding algorithmic play.


I feel like pinterest is being a "placeholder"/"fallback"/"default" of the web. Each time I encounter quora or pinterest search results I am getting the feeling I was looking for something that does not have good online answer/content.


Did you look beyond the Quora / Pinterest / other SEO spam results? There's good content on the web for virtually any query, it's just often buried under mountains of SEO spam results.


I did. Usually I am not clicking the pinterest/quora links and I am left with nothing.


“Deconstructing Pinterest’s reverse-image-search SEO growth hack”

https://www.rankscience.com/blog/pinterest-image-seo-growth-...


Ugh, this is bad. Makes you wonder, why they let this happen. Do people at Google own Pinterest stocks?


Yeah, Google has done so many things to thwart so many kindS of bad SEO (and sometimes even good SEO) that it's really hard to believe they've just innocently dropped the ball on this one. At a certain point you just have to conclude that they want the search results to look like this. "Oopsie, we just accidentally let one bad actor trample on this valuable 'real estate' for years. Silly us." Yeah right.


I think search at Google isn't getting the best and brightest anymore. It used to be the only product at the company, now it's just the oldest.

I also think there's a disconnect between how new algorithms are evaluated and real world results. Reverse image search is total trash now. Mostly returns annotations "red, dress, girl" instead of the pages where I can find the image.

Unless it's a brand name product, then it's exact matches.

So you're getting less than excellence in engineering, plus influence from shopping and ad dollar focused PMs and not much care is actually given to the search experience.

And it shows


> Do people at Google own Pinterest stocks?

TIL Pinterest is a publicly traded company (NYSE:PINS).


Oh wow :( I'm a bit scared by this, because of what it means for smaller competitors, who can't afford to pour investor dollars into hiring a whole team of software engineers just focused on SEO driven growth


Pinterest is very good at SEO and does some things that I have found to be quite interesting.

I posted this before in another post a while ago, but it is still relevant:

“For one of my boards that ranks #1 in Google on some searches I've found that the page Google indexes is quite a bit different than the one I see as a logged-in user.

One of the differences is that they display the text content associated with the pin. This is also used as the image alt text, but then appended with a bunch of keywords.

They also link to other people's boards which have names related to the images so it looks like "tags", but I have the feeling it is probably a mix of keyword stuffing/linking to other content for Google to follow.

The page title is also adjusted to include something like, "237 Best ________ images in 2020" followed by the board name.”


I search lot of academic suff like medicine and programing.

Sometimes I search for image results, and my search results are polluted by SlideShare and Slide player

SidePlayer is an ad infested site which also pirate PDFs by crawling for open PDFs and automatically uploading to their website

I don't understand why Google gives priority to such websites.

I believe including pinterest and sites like SlideShare and Slide player in our search results is what google should allow us to opt in not something we should opt out


Pinterest search results are so frustrating. There's a page that's close to what I want - but I can't get to the text ... I don't want a picture, I want to read a document. Seems like something is off, cuz it's crowding out sites with real content.


Another example about how google is broken. It has great difficulty distinguishing noise from signal.


It's also another example of how modern tech companies are broken (or lacking ethics).

Currently, many big tech companies do basically whatever they want. Facebook and other ad companies have been caught knowingly defrauding ad spend customers, Amazon has been knowingly permitting counterfeit sales for years, Uber has been ignoring regional laws globally, Airbnb was (and may still be) ignoring or hiding data that illustrates their connection with increased housing pressures, and on and on.

Doing what you can get away with (because the cost of the penalty is lower than the profit that results from the bad behavior) is not ok. It doesn't contribute to an improved civilization, it pushes civilization back down toward a zero-sum, selfish survivalist mentality. I know it sounds corny or hippie, but if people can try to better balance their own desires against the good of the community, the end result will surely be better.


The economy is meant to be built so that anything harmful to the rest of society is either unprofitable, or criminal (i.e. theft).

Obviously we have a new frontier of digital stuff, but the law and economic incentives are still ignorant of it.


There are plenty in power now who would disagree with your definition of economy.

Sometimes law and policy is ignorant of technical concerns, but quite often it is aware and simply has a more selfish agenda. Prime example of this is modern US copyright and patent laws and enforcement.

As long as elected officials have to raise funds to campaign and win, elected officials will be beholden to the people who financed them. If politicians didn't write special laws and loopholes for corporate special interests, they wouldn't last more than one term. They would be replaced by the next challenger who would take special interest money and then "pay it back" via special laws.


Is it really difficulty, or indifference? I think they could do better, but as a near-monopoly in that space they'd rather spend the money elsewhere.


Sounds like something Yahoo might have said when they declined to by Google. Maybe it’s a good time to start a search engine.


Pintrest has the worst UI/UX of any website. It is a cancer of the internet, contributes nothing and allows re-aggregation and front-running in Google search as a feature. It baffles me why people use this piece of shit.


An image bookmarking app that can suggest similar images is a genuinely useful thing. That allows a user to rapidly collate a set of images based on some theme (so for example I use it for design/illustration projects). Despite the UI, it works, and I can't think of any other apps that do the same thing and have a better UI, so :shrug:.

However I'd still agree with you 100% on its cancerous nature: it's an app with a limited, specialised use (emphasis on app). That app leaking and pissing all over search results is not at all good and, as you say, contributes nothing for most people. By analogy, if I have some todos saved in a todo app, what would be gained (or more tellingly who would gain) by making my todos appear in public search results? Best answer I've got is simply that advertisers want that data, and want users on that app to advertise to (so, fuck you, users of search who were expecting useful search results).


Wanna start one? There is an interesting overlap between submission of links (HN/Reddit/Lobster), bookmarking them (Del.icio.us), image based collections (Pinterest), and push-pull feeds (RSS/Feedreader).


Yes, I think that would be a good project.

I think the major hurdle is that the thing that makes it useful is an extremely large bank of images, and ability to share within the app. So that implies a social aspect, and then that needs to be gated off from search results

This has existed several times before -- this is very vague because I'd forgotten about until now, but the one I used to use in the early 2010s (?) was an invitation-only site with very simple UI, almost bare HTML (similar to pinboard), a weird name and a logo that was I think a gorilla. Used to have very high quality results, albeit highly focussed on art/design, with a small, dedicated user base

Edit: it was FFFFOUND!, shuttered in 2017 (Pinterest seems to have killed most of those sites, so how does one go about breaking that monopoly?)


This. Imagine working for Pinterest.


I often check my images to see how well there are appearing in the google search. Many times I have found to my delight one of my images clicked on it only to find it leads to another picture on Pinterest and is a completely different product and no link to me or my artwork or product. It is very upsetting that original source takes second place to third parties and these pins are vanishing and being replaced. I have no idea how they are doing this but it is making using Pinterest to show my original art and work a place to consider complete withdrawing from.


I solved this nonsense by installing a plugin in chrome that automatically adds -site:pinterest.* to all searches. Almost living in bliss ever since.


Care to share? A convenient, trustworthy plugin that lets you exclude sounds ideal. Anything for Firefox?


Unpinterested


For two reasons:

1. When people see a search result from Pinterest, they tend to click on it more than on the results from competing publishers.

2. After they click on it, they tend to engage with the content more than they do with the content from other publishers.

This is not specific to Pinterest, it's how modern SEO works. You'll notice that Pinterest doesn't dominate every search result in the world, but predominantly the ones where the user's search intent is more aligned with Pinterest's strengths (product discovery, visual stimulation, pet/animal pictures, art direction, etc.).

I think what people find confusing is how can Pinterest compete with other search results that have far more text? Well, it's not about the amount of text at all, it's about satisfying the search intent. And when someone types in: "what should I wear for my wedding," images are better in answering that question than text (a picture is worth...). As Pinterest's success in SEO teaches us, there are many more similar instances than we would have expected.


If Google decided tomorrow that Quora and Pinterest is spam and should basically never be in the top 10 results pages, couldn’t they just fix it? What’s the worst that could happen?


They would be sued. Pinterest and Quora don't actually care that their actions make the web crappy for us all. They would stand on their high horses and complain that google is treating them unfairly. Normally I would assume this problem is because all the cool kids at google are working on more interesting problems but in this case I honestly believe they can't make a change for the betterment of the majority of their users.


Does google have a legal obligation to be fair or transparent? I thought what made google tick was the army of manual and arbitrary blacklisting where SEO cheating like e.g showing a full article to the google bot and a “subscribe to read the full article” to everyone else was way over the line that could lead to a ban


DoJ, which is in charge of the on-going antitrust lawsuit against Google, will be very excited about that change.


John Muller (from Google) says about Pinterest:

I'd argue there's a lot of content on Pinterest -- even if it's not a collection of 3000 word blog posts. Sometimes images, even with minimal textual content, can be exactly what people are looking for. Not always, and sometimes we get it wrong, but it's certainly an option.

https://twitter.com/JohnMu/status/1260171380580024320

========= Regarding Pinterest, while I'm personally not a user of the site, there is a ton of content there. It's not 3000 word articles, but you don't need that for search anyway.

https://twitter.com/JohnMu/status/1316668506918866944


And the Pinterest pages have no content! They've beaten Google or Google doesn't care anymore, either way, plummeting result quality is a strong signal that Google Search is ripe for disruption. Ten years ago, Google would've punished all Pinterest properties for this.


I do not know whether Google gives Pinterest preferential treatment. That said, I'd like to know: What are you searching? Results will be different depending on if you're searching "cute bedroom ideas" or "epistemology in pre-Socratic Greece."


Ha good question! I run a company that builds visual search engines and one of the sites I build and develop is a colour search engine for lipsticks. One of the main ways people discover it is through Google searches so I was studying which sites rank highest for queries like "nyx budapest lipstick dupes" etc. So yeah, for sure, queries for stuff that lends itself well to visual curation will be full of Pinterest results unlike stuff about Greek philosophy.


Give it a few days and this page will be the #1 hit for "pre-Socratic bedroom ideas"


"epistemology in pre-Socratic Greece." has a few results from Pinterest.


i specifically hate how they hack slap a login wall on images, and dont allow you to right-click+save on mobile. they're intent on breaking the mobile web for their own self interest.

if you're going to SEO yourself to the top of google, good for you, but please dont use google as non-optional lead capture for your app that I don't want.


update: i wrote a personal google proxy to tweak google results how i want them. managed to get it recognized as a search engine (on my android chrome) via opensearch. not sharing the code for obvious reasons https://twitter.com/swyx/status/1342625544320339969


I’d love to be able to block a domain from showing up in my search results.

Pinterest being an obvious offender, but I can imagine people wanting to block certain news sites, or Fandom/Wikia sites, or all sorts of link farm spam sites.


I still remembered Google Panda update in 2011, destroyed many low effort SEO farms.

I wonder why Google stopped fighting this good fight anymore.


There’s a great Chrome extension that will remove those from search results.


  google.*##a[href*=".pinterest."]:upward:(1)

  google.*##.g:has(a[href*=".pinterest."])
Add to uBlock Origin. in the "My Filters" tab. This will completely block pinterest. No need to use more extensions that can access your cookies and history.


My girlfriend and I watched Interstellar the other night. She had a few questions at the end and I remembered there being some great infographics that would help explain. So I reached for Google Image Search and the results were dominated by Pinterest. It wouldn't have been so bad if I could've viewed the original full-size image - but for that I would've needed to create a Pinterest account which I absolutely didn't want to do. After that I gave up and drew my own with pen and paper.


I occasionally do copyright searches for owners of images (it's a thing in online groups where "do not post images without permission" is a thing) pinterest is full black hat when it comes to image ownership. It owns nothing it has on its site and without an account, there's no way to trace ownership using its links. And usually there's so many duplicates of the same images (without credit) that it swamps all search results.


Nice timing on this question - I'm actually deconstructing a website mockup (PDF) this morning and needing to find the originals of the images the graphic designer (who is off on holidays) used. It's not just the dozens of Pinterest copies, but even Medium or private blogs that are using an image and still ranking higher.

I find it disappointing that Google doesn't apply more authority to sites like Unsplash when it is clearing the original source.


Google hasn't returned the best results for years. They return the results someone paid to put there. Either through direct paid ads or through heavily SEM/SEO influenced results for everything except the most esoteric topics.

We actually ended up having to do defensive SEM/SEO on bing simply because of how cheap it is to game results there, just because they're not google (no one does it except scammers and people addressing scammers).


Just wanted to come here to agree. I blacklisted pinterest (and some other garbage content with forced registration for the purpose of data collection, like Quora) long time ago using a browser extension. It is particularly annoying when searching for images. I never clicked on pinterest either and wonder why Google is still showing me these results.

I wish they would natively allow blacklisting specific domains.


If Pinterest is one of your main competitors for search engine results there’s a good chance you’ve clicked on them plenty of times to check out what they’re doing. Google is going to personalise your results on that and they will think you like Pinterest. I personally practically never see them as a result.

Been a while since I’ve done any seo but iirc there are tools you can use to try to get the most objective serps.


For people saying Pinterest has mastered SEO and that's why they're at the top, what exactly are they doing that is so effective?

I'm not understanding how individual Pinterest pages or users would gather that many quality backlinks for example. Do they have quality backlinks? If not, why is lack of backlinks not hurting their rankings?


Could you please post some examples of search queries (text or image) that are dominated by Pinterest?

I share the sentiment in other comments that search engines are (deliberately?) allowing SEO abusers to degrade the user's experience. I noticed that for some queries (Like "How to brush teeth"), there are more ads on the first page than results! They're marked, but not in an immediately-visible way; I'd be most people integrate them with search results mentally.

I started throwing together a search engine about a week ago to address these concerns, and put it online yesterday. (https://www.pageref.org). I'm deliberately penalizing SEO abusers, and promoting websites that have high-quality content. Running custom searches on these sites in some cases based on keyword.

In a lot of cases, I'm throwing in search queries, and going through each result one at a time, and categorizing them; eg penalizing sites that use clickbait or scammy ads, that are low-quality but show up high in results etc, or probably aren't relevant, but are highly SEO optimized, like Pinterest.


>>>Could you please post some examples of search queries (text or image) that are dominated by Pinterest?

These are all image searches "sci-fi combat exoskeleton" -> 7 of 24 top images are Pinterest "zen minimalist clothing" -> 8 of 24 "grav tank concept art" -> 9 of 19

Like many other users in this topic, I run most of my G-Image searches with "-pinterest". I use DDG as my primary non-technical search engine, typically only switching to Google for images, trying to solve error messages, or researching frameworks/APIs/etc.


Thanks! Can rep


How much of the bafflement stems from the fact that Pinterest's own audience is overwhelmingly women, while the general search audience is 50/50 and the HN audience is, judging by the character of the comments, 99% men?


FWIW I use a greasemonkey script called "google hit hider by Domain". I used it to manually permaban all pinterest domains when they appear and now I almost never see a single pinterest result in my searches.


You can add:

  -site:pinterest.com
to your search to exclude pinterest.


Should be pretty simple to make a browser addin that adds “-pinterest” to search queries (I very often add that and search again, if I get a Pinterest result at the top).


Is there a ddg bang for search without returning Pinterest? (I mean I think -pinterest should work but maybe fewer characters, and I’m not even sure “-“ works anymore.


I am curious how your site accounts for different screens representing colors differently. It seems like it would be a hard problem to solve for this use case.


Wondered the same about Amazon as well. They show up first for most products.

Are those organic results? Or do they have a kickass SEO team that games Google’s algorithm.


Google has been rewarding a lot of the top sites that have the highest trust and authority lately. Pinterest being one of them.

This has been going on for a while now.


It does dominate image search which sucks. I instinctively add "-pinterest" to my search. I don't have this problem on text search.


This complaint features regularly on HN so [my assumption] it has to be wilful blindness on the part of google?


I always thought Pinterest was Google's site and that's why it was coming up so often.


I strongly dislike pinterest for this very reason. I wish I could block it from search results


Why does Google let Pinterest dominate their results?


For me this is an instance of Betteridge's law of headlines: "Any headline that ends in a question mark can be answered by the word no." :) I very rarely see Pinterest in my search results.


Are we using different googles?


I see Pinterest appearing disproportionately in image search.


I practically never see Pinterest in the results. I suppose it really depends on what you are searching for.


We might be doing very different queries :) If you search for makeup related content, Pinterest comes up a lot because people tend to pin and curate makeup related stuff there


If lots of people have spent time curating the Pinterest content, and searchers are likely to be looking for curated content, doesn't it make sense that Pinterest would rank highly in the search results?


You can search with adding `-pinterest` to remove all that noise.


Well, Google and Bing do favor big or older/established websites. And Google's search is trying to be smart when it clearly isn't.

I'm getting sick of my searches being "corrected", sometimes they even ignore the quotes.


Facebook users are linking a lot to Pinterest


I can't think of the last time I did a search and got any pinterest pages in my results. It must be more a factor of what you search for.


Are you referring to web/text search, or image search?


Web/text, but on the rare times I do image search, Pinterest results don't "dominate" my results as OP suggested.


I read the question as follows: how to rank so high like Pinterest does?

The answer is: you should not spend any second on SEO. It is time and money wasted if you are a SMB. Anything else, including taking a walk is better for your business than going into SEO optimization.

Your clients are people who buy your stuff. Search engine robot is not your client. After years of working with SEO it becomes clear for me, if you satisfy a robot, you don't satisfy your client. These are two different things.

The bright side is that when you will serve your clients, robots will catch up eventually. But it is search engine problem not something you should spend time on.

There is no magic bullet method to rank high. No special hack you can use. Everything you will read about SEO is smokes and mirrors. If you have million dollar company and did everything else right - sure, you can throw hundreds thousands dollars into SEO and pray search engine devs won't change their minds.

But as a SMB company, counting each dollar, just leave it. You won't win over search engines and people with bulk of money.

Spend your time and effort on clients. It is the only way to spend it right. If search engine will change mind - so be it. You won't be dependant on it. And your clients will come to you because they love your product, it is stronger than being number one in a twisted search ranking.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: