Where do you start with this one? But okay, here goes:
1. The post assumes that the search result quality is not good. This simply is not the case. Users say the search results are good. Could they be better? Sure. Are they terrible? No.
2. The post assumes users want transparency on the search algorithm. This clearly is not the case. The whole reason that google became what it is today is because of the lack of clutter and good search results.
If you remember Altavista then you remember users constantly having to fidget with all these knobs in the advanced section to get relevant search results. Lycos, excite, etc weren't much better. I remember using Google for the first time and it just worked.
3. The post assumes that users want deep customization. This again is not the case. Users want less clutter, less knobs, and a faster search experience. Google is delivering that.
4. The post assumes that any startup with a better algorithm can beat google. However this is not the case. Google has deep search integration with mobile providers such as it's own Android OS and also iOS. it also has it's own browser (Chrome) and has paid deals with Mozilla. Obviously, both browsers default to Google when you type something in the search field.
Not to mention, the leading IOT devices all connect to Google to find results when it needs to search the web. And for safe measure, Google has come out with its own voice assistant device, Google home, to safe guard against any shenanigans from Apple's Siri and Amazon's Alexa that might happen in the future. It is heavily subsidizing the price to make sure it has a strong horse in the voice game.
Integration aside, it also has scale. It crawls most of the internet several times a day. That's no small feat and it is quite hard for a startup to just replicate that with funding. I mean, even that won't cut it.
Google is now even using machine learning to understand relevant pieces of information on a webpage so when you query it by typing on your keyboard, by voice, or future mind control it delivers the answer you were looking for.
The only valid point the post makes is that Google has squashed competition by buying any startup that remotely could threaten it's position...for safe measure. I'll give the author that.
(1) My impression from using Google is that it's really great for technical / scientific first page search results, but pretty horrible otherwise.
(3) No, I'm pretty sure users want more knobs or at least filter options. Hell, even a simple domain blacklist feature would be amazing so that I could get the woowoo psuedo science cruft -- which has been SEO'd to the front page -- out of my results, so I didn't have to put any mental effort into visually filtering out multiples from the same site. Hell, they could even collate results from the same domain into a single grid folder to optimize the time I spend searching through the search results.
Don't conflate Google's lack of innovation for an optimized experience simply because it's Google. I love Google, but there is a massive amount of improvement that can go into their search user interface. Same for every other search engine, but Google is the leader in this space, so they get the brunt of the criticism, and should they change, then others can follow suit.
> No, I'm pretty sure users want more knobs or at least filter options.
Nah, 90% either don't or wouldn't use them if they were there.
Sure, if you ask people directly if they want more options, they'll say yes, who doesn't want more choices? But that's a different thing from actually using them when they're available. The vast majority of users for a utility service like a search engine just go with the defaults.
I once had users begging me daily to implement 2FA, being so dramatic you'd think it was a principle they'd die for. It became a huge issue in the chat daily. How can we take this site seriously if there's no 2FA? We have no choice but to leave to <competitor> if you don't have 2FA!
Well, a year after we implemented 2FA, I queried the database for users with 2FA enabled, 3 people had it enabled.
Aside, only somewhat related to the previous story, the idea that users want a bunch of knobs and levers is just an example of the HN delusion where HNers regularly think they represent the population, like the person using the Lynx browser.
(1) Is it really that hard to implement 2FA, and
(2) Did you just implement the same insecure SMS nonsense that's been getting people's accounts compromised for a while now?, and
(3) Did you bother putting anything in your UI saying "Hey! Over here! Look at this new feature!"
Having an advanced set of tools means that anyone can build a simple interface on top of it, providing a subset of features. It's called the facade pattern. It's present in nearly all software systems, in one form or another.
Don't let your supposed experience as a professional techie give you the false impression that you know everything there is to know about tech.
The person/people on the Linux browser are hilarious. Half the time when someone launches a new site and shows it to hn, they chime in complaining it doesn't work on lynx.
The first few times I saw it, I thought they must have been joking or trolling.
I don't even have time and budget to make my site work well on older versions of IE. It would be insanity to optimize for the 12 people that use lynx.
If your page isn't at least readable in Lynx, then it probably isn't readable by screenreaders or crawlers either. Lynx users aren't expecting pages to be "optimized" for them, but it's not unreasonable to expect text content to be accessible textually. All this whiz-bang JS frippery is shaping up to be Flash wesbites, round two; it was bad then, and it's bad now.
Even Facebook used to work with Elinks (don't know if that's still true).
> Hell, even a simple domain blacklist feature would be amazing so that I could get the woowoo psuedo science cruft -- which has been SEO'd to the front page -- out of my results
You can put w3schools in /etc/hosts to an IP like 127.0.0.1. You still have to visually parse the search results, but if you accidentally click, you’re not giving a pageview by mistake.
1. Google has become less intuitive to use, but given the right queries it will return the best answers on the first page which the others will not. Almost always the first result under the ads is what I was looking for and that has improved. My searches are programming and travel and both are spot on, but only when formulating things in english-logic speak and using the right nomenclature. If not, you still get related topics and usually still better than the competition, but meaningless in the intent of the search. If I had to use ddg or bing or such for work, I would lose a great amount of time going through pages; I cannot remember when I went to page 2 on Google. For programming questions, you can be sure if it is not on page 1, it was not answered. Page 2 and further will contain scraped SO results for instance (I would like Google to be more firm on that but he).
There is an insane amount of content on the Internet which will not be returned in a query, no matter what you enter. More than half. More than 90%. Probably even close to 99%. Yes, you will get better results than the competition. No, it is not better by much.
And I don't go to page 2 anymore because Page 2 will always have garbage on it. I remember in the old days, in the 90's, I could search and get 5 or 10 pages of unique pages. They might not have all been entirely relevant, but they might have been interesting reads nonetheless. With today's Google search, I get about 3 or 4 typical relevant results, often from the same websites vomited out from the top 50 from Alexa 500. And then a lot of spam, computer-generated web pages after that.
There is no hope for anyone who wants to independently publish "a web page", anymore. I realize that this isn't something easily solved due to the fact that the Internet is vastly more populated now. But I don't think the system that we have in place, with Google, is anywhere near the level of sophistication that we can achieve as we approach the 2020s. We have this narrative that Google is such a vastly powerful collective, and that this is the best that they can come up with, and therefore that's what we have to work with, as Internet users. My post was not to show that I have all of the answers, but to illustrate the fallacious reasoning which deters people from even attempting to solve the problem.
Folks, as long as we continue that mindset, that's how things are going to continue to be.
I believe you are right and I am very careful to say things about curtailing search results. The problem ofcourse being that I am a weird person who’s mindset is set mostly in binary; when I search for something, I will find the answer or not. Life does not work like and possibly a more nuanced approached will be better. However I am also deterred by that same life; people want simple and want it now. Like I read that we walk on the back of our feet instead of the front because we are lazy, we levitate to anything that is easier. Even if it is worse, even for the individual.
But in general I agree with your rant; please experiment! I just do not believe it will take hold easily. And they article seems to suggest so.
IMO Google is unbeatable as long as nothing changes, if you really have to recreate Google as it is today, it will cost too much and you will never make it.
But if there are technological shifts that invalidate the investment Google has made, then there is a chance.
I think you're spot on about why Google is investing in the Assistant, but I think you underestimate the threat from Alexa. If Alexa gets a foot hold and manages to use the fact that you're having a dialog to deliver commercial queries better, Google is in a lot of trouble.
Vertical-specific commerce sites are also a threat, if you no longer search Google for insurance, but go directly to an insurance aggregator, Google's search tech is irrelevant to capturing that advertising revenue. I think Craigslist is undergoing this problem since they've basically refused to invest in better UX, vertical specific sites are eating their lunch.
I just don't see Alexa making huge inroads for shipping. It will be/is great for recurring purchases of stuff you already buy on Amazon, but people were already going directly to Amazon for that.
But using a voice system to buy new stuff just isn't a good experience. It's like the old days of catalog shopping with a phone except there's no catalog.
People need to pull up and see, look at reviews, watch YouTube review videos. Audio only just doesn't do that and is a step backwards.
I don't even use Alexa for reorders because I want to make sure I'm still getting a good deal everytime I order. And for those that don't care about the deal, they can just setup subscription purchasing.
Some people need that, sure. I think Amazon may have a trust problem, but I've brought products based entirely on the Wirecutter/sweethome reviews, and I can easily imagine talking to their bot about what microwave they recommend. None of this works well now, but if Amazon builds a hardware moat, Google may not have a chance to compete with better software.
5. Plain old capital-intensive advantage: local server farms, enabling low-latency search (and "suggest") results, which been found to be very important for users.
One can interpret this (and android) as google desparately spending money whereever money can give some competitive advantage, no matter how small, because google is acutely aware of how easily users can switch search engines. It's not a super clever strategy, but just: what else can you do with all this cash?
I use DuckDuckGo as my primary search engine. It is good enough. But their algorithm is different. It took me some time to adjust for that so my searches works like they did with Google.
Which point out that at this point not only Google has a good search algorithm, but users are trained to help it do better. A search startup has to overcome that as well.
Can you give some tips about how you adjusted your search queries? Whenever I tried to use DDG I wouldn't be able to get results from it that were as good as Google's.
This is very subjective and I am not sure that it even matches the reality (it could be that within the last 6 months the search engines changed), but with DuckDuckGo I stopped to use verbs at all. Another thing is that at least for searches about IT Google prioritise recently posted/changed pages more so with DuckDuckGo I often added 2017 to get the latest hype (soon to be changed to 2018 :) ).
Whatever it was at the beginning of the switch I used to use Google on a daily basis when DDG results were not good. Then in few months it pretty much stopped.
I remember being a little creeped out when Google first started automatically localising queries, but when I tried switching to DDG earlier this year I realised that I've become utterly dependent on it. I constantly had to think about whether to append my city or state to the query and even after doing so would often get American results. (For example, searching for anything relating to the WA government still turns up information about Washington, even when the country is set to Australia.)
> For example, searching for anything relating to the WA government still turns up information about Washington, even when the country is set to Australia.
This is so annoying; I don't understand what the country toggle is actually for, since it doesn't seem to correct this?
I use DDG from Norway. In DDG there is a prominent checkbox to localize for searches there. Checking it gives the necessary localization even for queries in English.
1) The issue is not with search result quality for a given query. In fact, I agree that such quality would be difficult to match. But this is a very specific, very polished feature. It is by no means the only way to deliver a quality user experience.
Rather, the issue (and perhaps this is just one issue with many more I am unaware of) is that the search space of possible queries is limited to a small number of dimensions.
2) Definitely under no illusion that there is a demand for transparency on the algorithm. However, if that information is a successfully-guarded secret, nobody else is going to create an incremental improvement over that specific type of feature.
3) You have two incorrect assumptions here. Not all users want the same thing. But more importantly, there is absolutely no reason why more customization means that the UI must become more complex. A company like Google is smart enough to see that. If you have more customization, it can be offered through an API. Even if not offered through an API, you could see open-source browser plugins to create a custom page, with no clutter and no knobs. And likely a fast search experience with some effort.
4) Absolutely false! The point of my post was to illustrate that the failed strategy of "imitate the UI with a better/different algorithm under the hood" will never work.
Your point regarding scale is a good one, and it is exactly part of the reason why I believe that peer to peer blockchain technologies will be the likely place for this dethroning to occur. Users can collectively contribute their own little piece of the network, sometimes even being paid by doing so (refer to Steemit for a proof-of-concept). There are a lot of interesting ideas in some of the Ethereum distributed app whitepapers. Most of those ideas will never be successfully built, but there are enough of them where we will see some successes. Then the industry will learn from the mistakes that were made, and we will see a new paradigm. I think this will happen rapidly over the course of a few years. I already have specific conceptual ideas for how a search engine might operate (URLs can be associated with a reputation rating, various tags, and so on, and not bound to a centralized server).
1. How else would you like the results to be delivered? It is delivered in browser's search bars. It is delivered by visiting Google.com. It is delivered by voice. It is delivered by mobile devices. It is delivered inside my Nintendo Wii, XBOX, PS3/4, and my Nintendo Switch. It's delivered on my crappy Vizio TV via smart apps. It's even delivered in my car. Google search results are everywhere.
2. I think that is kind of the point. However, you assuming Google is being evil by keeping it a secret. That's not fully the case. A lot of SEO companies would love to see how things are ranked so they can beat it on behalf of their clients.
3. They already have an API and if you really want to customize the query, you can add all kind of query modifiers in the UI.
4. As a developer myself in the crypto space, blockchains make a poor curated database for the very reasons why Google keeps things a secret. Anyone with money can pump the reputation of an entry on the blockchain to get better rankings thus lowering the quality of results.
Second, most blockchains can't handle the query load.
Three, dapps cost money to run. I can see a market for users submitting entries. If there are users using it, it might catch on in the submission department. However, this brings me to my next point.
Four, the chain would not have all the websites on the net. Kind of expensive to get it all on there and not scalable.
Lastly, someone has to pay to run the query side too. Would users pay to run a search query and get gamed incomplete results? No.
I can't respond to all of your points right now but I am interested in your comments about the blockchain. As a developer in the crypto space, I'm surprised that you're apparently not aware that reputation doesn't need to be tied to stake in order to exist. But even more surprising is that you're making an argument that people with money can abuse the reputation system, when the very reason that it is tied to stake is to make it prohibitively expensive to abuse that sort of thing. Decentralized tribunal systems are in the works (I designed one myself in 2011), to further make the network more resistant to such things.
Regarding your comment about blockchains and query load, you also seem to be unaware as a developer that blockchains are a source of truth, not a front-end. A user request does not cause a query to the blockchain. A network of server nodes can exist, with cached results. The servers themselves can have their own reputation system, or the entire front-end experience can be managed by a single centralized entity without compromising the use-case of decentralization. Specifically, if the owners of the front-end were to act in bad faith, it would be much, much less expensive for someone else to spin up another identical front-end (most likely open-source as there would be a lot of early criticism if it wasn't).
Regarding expenses and costs, solving that problem is part of the hard work. If I were building the system, I would probably not start out with the entire Internet being the scope. Perhaps as a proof of concept I'd start with Wikipedia only. I know for sure that paying for each search query is a non-starter, as that has a psychological effect of being punished for using the service, when you want it to feel rewarding instead.
Right. Sure, you could cache it. I will give you that. But you have to index the entire internet for it to be interesting to anyone. The first Google did that. It indexed most of the pages out there.
Why? Because the instant I search once, twice, three times and find nothing. I am abandoning the system.
Blockchain aside, it doesn't matter. Whatever system you build has to be 10x better than the current system. Nothing you have said, blockchain usage included, comes close to getting there.
Nothing I have said comes close to being 10x better because I have not posted any design specs, proposals, or pitches for such a system. I certainly can't beat Google by 10x with some offhanded comments which were not the point of the original post. But some of the misinformation you are spreading about blockchain technologies contributes to the negative sentiment seen around here, and it had to be addressed.
To me your posts come off as the hand-wavy "just sprinkle in some blockchain magic!" sentiment that is the main cause of blockchain eye-rolling on HN, not the person putting you in the position of elaboration.
Mr. Gupta asked, in a respectable and direct manner, for me to elaborate. And I did. You, and many of the posters here, did not ask nicely. So I did not.
I don't know why but google results became mediocre/bad recently. There were a time when I could pick [good] google results in a blind test (for programming topics) between several search engines (Bing, Yandex). Now, DuckDuckGo can return noticeably better results for some queries than Google.
DDG is also easy to use as a shortcut - `!python import` for example to go straight to Python docs search results. With Google without ! I'd be hunting for the result from that site, because site:... is longer to type, and I probably wouldn't remember the domain anyway.
I agree with almost all the points apart from transparency. Transparency is not customization. Transparency is basically like, you search something google returns something, and then there is a link which tells you why google returned what it did. This is the transparency which is currently being pursued by the EU. That algorithm providers should be able to explain the results of their algorithm.
I'll come out and say it: Google's search quality is terrible. It's better than the competition, but that's a low bar. Most people do pretty good with Google simply because 1. their searches are "solved" by simply reducing the search space to pages that contain the combination of rare words in the query (e.g. "correct battery horse staple" is referring to the xkcd comic with high probability), along with the social signal of backlink ranking, 2. Google has trained us to use it effectively and so our expectations for search match that which Google can effectively deliver, and 3. All the meta-signalling that Google uses to turn obtuse queries like "that movie with the guy who does the thing" into the correct answer. But in general, if you can't figure out the exact words that are on the page you're interested in, the illusion fails and it reveals itself to be barely more than grep on steroids. Search is still waiting to be blown wide open and it very well could be by a competitor.
I suspected the author was dreaming when he said the lack of rich customization of the search algorithm is taking away from the end user experience. People don't want to learn about the intricacies of your search algorithm. You're assuming they care. All they want to do is type their question into the magical text box.
He also seems to not understand the value that search advertising provides. Search advertising is a win-win for businesses and consumers. If you see it as something parasitic that needs to be fixed, I can't take you seriously about "how Google will be beat".
Then I read the last sentence, quoted below. Sigh. Nothing serious here folks, carry on.
> I predict this will become more of a real possibility under decentralized smart contract systems such as the Ethereum blockchain, where value exchange itself can happen on a peer to peer basis (via a synergistic ecosystem of content curators, content providers, and content consumers).
Consumers are often in a "buy mindset" when searching for something. The phrasing of their search query can indicate it. For example, someone search for "cheap office chair" most likely already has their CC in hand. It's a search query that belongs as much on Amazon or Craigslist as it does on Google. So when it's searched on Google, the idea is that the market will provide more relevant results than an algorithm designed for ranking articles based around keywords and semantic equivalence of the content.
> So when it's searched on Google, the idea is that the market will provide more relevant results than an algorithm designed for ranking articles based around keywords and semantic equivalence of the content.
And the market does this in a relatively simple manner: by letting merchants bid on search keywords, and showing the site of the merchant with the highest bid, you force merchants to either lose money or be relevant.
In other words, the most relevant merchants to a search phrase will have the most money to bid up visibility for that phrase because they make the most money, on average, from users looking for this particular thing.
Good question, when the ETH contract drivel appeared at the end, I hope he is somehow meaning to pay users Ether to see ads. ProofOfView whatever.
People would fix Google if it had an API but it would mean censorship; I would add a ‘credible sources only’ button which is not good for the web but it is what I want. I do not care for traveldiscounters’ SEO or SO or Wikipedia scrapers; I care about the originals. That would greatly improve the experience, but in general it is bad for the internet because someone decides who is credible. Now there I see a place for Blockchain.
Forget about Ethereum for a minute. In the case of Wikipedia, no single company can produce the amount of content that they have on Wikipedia, especially with that reasonable standard of quality. Why is that? Wikipedia has a decentralized culture where principles are the main driver of decisions that are made. Wikipedia's collective Administrators, Bureaucrats, and Stewards do not compose a top-down management structure. It was not easy to do, requiring strong community outreach, culture building, and hard work. It is unlikely to be repeatable in the same fashion given that most Internet users feel they can find what they need online (the innovator in me disagrees with that sentiment, and I am very much aware that my opinion is a minority).
Ethereum, in my mind, is merely a way to help automate, standardize, and enforce similar systems. If it can be achieved without any sort of blockchain, like in the case of Wikipedia, then great. But perhaps the barrier to doing so is just out of reach without the help of the technology. Yes, having a central authority decide who is credible is exactly bad for the Internet, and Ethereum is a possible means to facilitate alternatives. What other alternatives would there be?
>Search advertising is a win-win for businesses and consumers. If you see it as something parasitic that needs to be fixed
Its absolutely parasitic and not a win for consumers. If it were a win for consumers then Google would not be so fantastically profitable.
Search results on Google are great provided you don't use a search query that indicates an intent to buy something. If you do that then the results turn terrible. Or profitable. Depends upon whom you ask.
Indeed - do a search for anything Google determines to be non-commercial (like "retrovirus", "contract for difference" or "banana") and they simply hand you off to Wikipedia. It's Wikipedia, and not Google, who really own the show when it comes to collating the world's knowledge.
Now that's a bit conspiratorial. Probably a majority of people searching for "retrovirus" clicked the Wikipedia article link. Google figured this out, and started listing it first. Nothing beyond Wikipedia fulfilling its role.
Definitely not assuming that the average users care about the algorithm, and you missed the point being made there: If that information is a successfully-guarded secret, nobody else is going to create an incremental improvement over that specific type of feature. That point was intended to lead into the greater premise of the post, which is that the conventional wisdom of competing with Google by going toe to toe, is not viable. I apologize that my writing is not as intuitive as Google's user interface.
Especially given the amount of vitriol I see on Hacker News regarding decentralization technologies (not surprising given the heavy ties to Silicon Valley's centralization culture), your condescending comment wasn't helpful to the discussion. But I appreciate you giving it a full read nonetheless.
I don't know if an open collaborative approach to search is really possible given that there's a whole SEO industry out there. If we could have the equivalent of white hat SEO folks help point out flaws and exploitable bits, then maybe?
I'm going to restate because I suspect miscommunication here... my point was that an apparent 'centralisation culture' is a result of building centralised tech being typically simpler/cheaper than building decentralised.
I understand that you used the word "typically" but Wikipedia is relatively simple (by some metrics), and operates on a relatively low budget, and is decentralized (managed by the users, not the Foundation). In fact, a company with Google's resources, working 100% toward building the equivalent of Wikipedia, would not have a guarantee of successfully doing so, with the entirety of their paid staff, let alone figuring out how to make a return on the investment. Perhaps IF they had done so, our Googlepedia would be an Encyclopedia Brittanica, with a search bar, and ads. And everybody would argue that there is no possible way for a competing business to earn more revenue, with a similar product. Completely missing the vision because Wikipedia never existed in that universe.
That is the power of decentralization, regardless of the naysayers and corporate shills who continue to dismiss the whole concept, by citing examples of alpha-quality solidity contracts. Ethereum, despite being itself barely beta-quality, and highly primitive technology of its kind, is still a disruptive technology that threatens the status quo. Creating an app that has the strengths of Wikipedia, but functions like Google, Facebook, or Twitter will be an order of magnitude easier with the help of this technology, and the right team. Maybe not enough to disrupt things outright, but certainly enough to be a large threat, and the next generation of that type of technology will have a near-guarantee to bring that disruption. Obviously there is a lot of vested interest with deep pockets, toward discrediting it. The resulting narrative, combined with the consumer culture/sentiment toward the products by the same companies, is what makes up the centralization culture. I think we need a catchier phrase, maybe even a buzzword to describe it :P
Don't get me wrong, I think many crypto enthusiasts out there are insufferable man-children with criminal records and an axe to grind with "the man". So blockchain's culture might be the problem in the sense that some of the users are giving the technology a bad name.
Sorry, I disagree with your Wikipedia example. If Wikipedia is decentralised then so is literally any other site with user-generated content/moderation. Wikipedia's technology, policies, financing and infrastructure are all controlled centrally. For me that example stretches the word 'decentralised' past the point where it's useful.
Wikipedia is open source content though, which was a big aspect of it being successful at the beginning (as in, many of us wouldn't have bothered to write content for it had it become proprietary).
Open-licensed content cannot be said to be controlled by one party, as anyone is legally entitled to create a fork. Thus, Wikipedia is not controlled by the people who run the servers; the infrastructure may be centralized, but the project (the compilation of knowledge, and the articles produced) is not. This aspect of control is what I said you were missing in your assessment, and it seems I was right.
Decentralization is not a binary characteristic. It is an architectural pattern that can be applied in varied contexts, varied degrees, toward varied goals. It is also a process. You can't decentralize something that is already decentralized. Wikipedia was a pioneer in Web 2.0. Smart contracts and decentralized apps are being referred to as Web 3.0 (So was the semantic web for a number of years, but the semantic web will likely be part of this).
Ethereum is a platform for creating decentralized applications, and it too has technology and policy that is managed by a central foundation. That doesn't mean that the ecosystem is centralized.
Zooming out, we can see that the Linux Kernel development itself has a centralized development model, even with a dictator in charge. But the Kernel is a platform for which anyone can create and ship a distribution on top of. As a result, I am able to enjoy the use of three or four distros (or more, if I wanted) that are most tailored to my use cases, instead of being forced to choose between Pepsi and Coke (Windows and OS X). And many people can get into operating system development as a hobby, with a much more achievable criteria to do so.
We can zoom out even further: The FOSS movement itself might not have existed without the relentless efforts of a few individuals over a span of many years. But from day one, the ecosystem it spawned was decentralized by design.
Going back even further, we can show that market Capitalism allowed for a rapid and diverse pace of technology and business development. Less barriers to entry. Less red tape. Obviously we have had governments, both big and small, to facilitate the economies of a nation.
What do all of these things have in common other than incorporating applied forms of decentralization? It is not just a buzzword. It is no coincidence that these frameworks have (or will) foster significant societal changes, mostly in good ways. That's not to say that the success rate is high for any given effort, but it is not a concept to be written off, especially not because of a lack of understanding of it.
The problem is that most people, even here, cannot visualize this concept in ways that they have not seen working examples of. Not a problem, not everybody is exactly a visionary in every way. But when people are actively trash talking ideas that they don't understand, without offering constructive opinions toward an understanding of it, then as technologists we ought to be taking a stand against that behavior.
Instead, what I see in many cases is that many engineers, for whatever reason, will write off certain technology ideas, believing that they understand it, when they do not. I was one of those people, for years, so I understand the mindset as good as anybody. Or perhaps they saw too many uninformed comments coming from non-technical crypto enthusiasts, and therefore have written off the idea. Folks, if this describes you, I encourage taking up the study of critical thinking techniques. The world deserves better than to have such a large number of engineers not giving the discussion a fair shake.
Now you've diluted your "centralisation culture" comment to "engineers, for whatever reason, will write off certain technology ideas" I can probably agree with it.
This was non-sensical...someone will compete with google because Blockchain? There’s no issue with current payment models, you don’t need a trustless peer to peer system to beat google, you need to significantly change the nature of what it is to search. If instead of finding q&a’s on stack overflow an AI took my question, looked at my code, and answered / fixed it, that would be a game changer. Using a different UI than google? Has happened, will happen again, not really relevant (google itself has revved it’s ui extensively).
What are some business models that could support a user-oriented search interface? Probably not advertising, if (big if) the new approach will return better search results than Google. Subscriptions have only worked for a few well-known services with exclusive content.
Would users be willing to learn a more powerful interface, if they consider current search engine results “good enough”?
Could Wikipedia or Archive.org address a subset of the search market, since they offer user-focused services? Archive.org already crawls a subset of the web.
There are probably many that I haven't thought of. It may be possible that it is next to impossible to create both a better experience for the users, and create more revenue for the company at the same time.
It doesn't need to have a huge amount of users, it doesn't need to get the majority of the market share, and it doesn't need to earn more revenue, or even any revenue at all, in order to deliver a better user experience. Such things are not mutually exclusive.
I believe this is most likely to happen with the help of peer to peer technologies, combined with a healthy synergistic ecosystem of content curators, content providers, and content consumers. Steemit is a crude proof of concept for such a system. In Steemit, comments and posts are scored with a reputation algorithm, and assigned a tag. There's absolutely no reason why web links can't be associated with an arbitrary number of contextually-weighted tags, and a contextual reputation algorithm.
Everything that Google does could be done in a peer to peer manner, with each user bearing a small piece of the responsibility. Wikipedia is a great example of this, with many users contributing to the system. No single company could ever produce the amount of content that the Wikipedia userbase has created. Even though Wikipedia uses a centralized foundation and centralized server, the editing culture itself is decentralized, with a set of principles being the main driver of decision-making.
I bring up blockchain technologies, much to the disgust of many users on HN, but only because I see those technologies as lowering the barrier to creating systems such as Wikipedia, through automating, standardization, and enforcement of some of the mechanisms. It also creates a guarantee that a central authority doesn't exist and therefore cannot be manipulated in subtle ways by those in positions of greater power - That is a big appeal factor for a significant number of users, even if it is not the majority.
>>> There are probably many that I haven't thought of. It may be possible that it is next to impossible to create both a better experience for the users, and create more revenue for the company at the same time.
Yes. Google cannot be beaten.
I don't think that any domain has more money to offer than advertising/marketing and certainly not easier. Then the real time auction and pricing systems that run Google are truly incredible.
Business wise, google is milking the perfect cow perfectly. There isn't a better cow or a better milking strategy.
Most people want the accessibility achieved through quick searches on their phones. The only users that would use a more powerful interface would be professionals. The professionals that would use it would be researchers, developers, and health practitioners.
There's no reason that a powerful interface can't have an arbitrarily-designed, simple interface built on top of it. That's how computer systems generally work.
I wonder what those professionals use today: google’s advanced search interface, niche site search, social media search, libraries, paid services, bloomberg, lexis-nexis ..?
As far as I'm aware, in the Netherlands wrt health: specialised sites. Google is all WebMD or unverifiable claims, or just doesn't go in depth. So you check your UpToDate, or pubmed, your country's/specialization's current guidelines, farmacotherapeutisch kompas for medicine (https://www.farmacotherapeutischkompas.nl/ - search for medicine and find dosage, interactions, etc). Google is often good enough for a reminder, but won't hold up for specific knowledge or minute details. So at most that Google the name of the service to search on there. Which is why I use ddg - if I know what website/information store I'm going to end up going to, I might as well use the !bang to go there directly. And you know which tools to look at because you're given them when you're studying.
As someone who tried to compete with Google in search, it strikes me that everything he brings up was also true, and well known, in 2007 when we started Blekko.
Cuil, SearchMe, and Blekko didn't fall into the UI trap that he outlines. None of us succeeded that well, but that's a different issue.
Well, it's always hard to say why something didn't work, but we certainly tried a very different user interface with our Izik UI, in which ambiguous queries got up to 200 answers organized into categories.
I'm curious as to how it failed. Did people use the site, and hate it? Did people join and then not return, or were those metrics unavailable? Were the operating costs abnormally high? Did the company receive a lot of users, but fail to receive enough revenue to cover even a normal amount of operating costs? Seems like one could narrow it down with the help of today's analytics tools.
I'm also curious if you have kept up with other web search technology efforts since then or whether you moved away to other things. There's definitely some things that don't add up.
To get people to switch the new solution needs to be 10x better. It’s not clear how you could do this without Google’s day network effects. Someone like SoftBank would have to go out and bets $10b on a search startup, but even then, Google is not going to be outsmarted by a new algorithm or interface. We’d need to move to a whole new mode of computing. Mobile has been the biggest threat yet.
I switched to bing long time back and haven't felt any difference. They even copied google look and feel to great extend. I did it after google started showing more details in my search timeline than i wanted to see, with it's integration with gmail with which google started scanning my emails for ticket purchases, product orders etc and whenever i searched that product - it says you have purchased this product with order tracking number blah blah.
I am sure bing/msft also does everything they can to 'understand' me, but at least they don't see my browsing history or emails.
Now back to beating Google - I do believe it's possible, but Google will soon catchup with any potential competition. They will catchup because it's their bread and butter and they have perhaps the worlds biggest concentration of technical personnel. The best way to beat any product/company is to be considerably better than it in at least few areas, given the scale of web indexing, that is a very very tough job only few companies can pull through.
Microsoft has/had a good chance to beat Google, but Microsoft is considered a bigger evil/corporate than Google, so they have a slim chance to emotionally overcome Google, not to mention technical capability. But they can perhaps open source bing in an effort to beat Google, provided they have the will and moolah to do so. When I mention open source, it's not the code, but the underlying search index, exposed as an API, free of use for anyone/any product. That will dramatically shift the emotion as well as the usage towards bing. I even mentioned this during an internal bing meetup event (I am part of a bing beta testers group), but of course no one listens.
A startup can also try to beat Google in their wildest dreams if they can associate themselves with the younger crowd like snapchat did and the younger crowd somehow finds it hip/cool to not use Google and use that service instead, and provided they get a decent web search data/index from established competition like bing or they manage to index themselves. These are few of the possibilities.
I don't know if the majority of consumers would care in the slightest about Microsoft making their index public. Maybe among certain crowds like this, but what would it gain us? I couldn't just send a PR to help improve it; the only benefit I can see is transparency and perhaps a fun, maybe even useful visualization.
Also, the image of a "hip" Google competitor like Snapchat seems tremendously unappealing, personally.
> I don't know if the majority of consumers would care in the slightest about Microsoft making their index public.
It's not just about consumers, but a plethora of other startups who can then use the index to build something creative on top of it. For e.g., you currently have bing/google customise certain type of websites with custom view for quicker access (IMDB, Wiki etc). However there is a limit a big company can be creatively about this.
Imagine a search engine where any user can submit a custom widget under a market place and consumer users can then install that as a plugin. For e.g., stock brokers can install stock widget, programmers can install document/stack overflow search and so on as a quick widget to see the top result without opening the link.. since it is community maintained, it will be proactively maintained by the community itself. That alone can topple something like Google if there is enough momentum (E.g. see VSCode overtaking Sublime text.) That is just one possibility. And it can also improve the search results as users may voluntarily come up to improve the result accuracy.
Other possibilities include voice based search, gesture search, integrated search within any mobile or smart watch app etc. With current licensing model, other startups have severe limitations in obtaining search results from the wast internet. This could explode and perhaps take over everything.
Ah, that makes much more sense now. You want to enable a community of plugins/widgets/apps built on top of the index. That's fascinating. I feel it would probably require some centralized base set of widgets similar to what Google already provides. That way the average user can just type "5 minute timer" without having to manually install the widget first. Quality/security outside that base set might be nightmarish. People put a lot of stuff into search bars.
May I remind you about when it was hip and cool to use Google? Google's tried to keep that image going as long as possible, but as a giant corporate behemoth, it's hard to realistically expect that to hold up forever.
Is it possible? Absolutely but it won't last very long. Google would have learned from Yahoo's mistake. They will acquire a company if there is even a whiff of serious competition. Whether that better algorithm will end up in Google search is another matter.
It feels like this Quora answer explains why Google is successful, but doesn't really discuss or provide any compelling evidence as to why the author thinks it's "becoming more and more likely to happen soon".
Also,
> I predict this will become more of a real possibility under decentralized smart contract systems such as the Ethereum blockchain, where value exchange itself can happen on a peer to peer basis
"Number of people who tipped the author for this content" could be a very interesting search signal. It costs you nothing to link to someone. It costs you money to tip them.
Yeah, but if it becomes known that that is a search signal, it effectively lets people purchase their way up the search rankings by paying other people to tip them. You'd have a much smaller pool of legit tips to overcome, because so few people pay for content, before you could become the most tipped article for any given topic.
I have tried not to use Google, but I always come back.
Results from Altavista, Bing and DuckDuckGo are so much worse.
Note, it's not that Google has personal information on me. I'm not logged in, my cache is cleared, I use a VPN and google.com/ncr when it was available. Not sure how they do it. Perhaps their tracker sees how long people have visited a page and then they give those pages higher priority.
Perhaps other search engines can do a better job, but they had over a decade to improve their results and they are still far off.
In my experience, the main-difference between Google and other search engines, aside from personalized results, is that Google tries very hard to interpret your query.
For example, if you search for some programming topic, it will generally pull results more towards the top, if they reference the current or a recent version of whatever programming language/tool you're searching for.
That helps, if you just want to type in "how do i do this? [programming language]" or similar.
In other search engines, you'll instead first search "how do i do this? [programming language]", then possibly not really get any good results and then instead search for "how do i do this? [programming language] [current version]".
This is how Google worked as well, some few years ago. Knowing what keywords to type in to get the right results was an actually valuable skill ("Google-fu"). Today's Google is instead designed to minimize the need for skill in phrasing your query. Great for the average user, but in my experience, if you do still possess Google-fu, then this interpretation actually hinders you.
With other search engines, you get the results that your query asked for. You know immediately when your query was ambiguous or just bad and can adjust accordingly. And with that, you can more easily narrow down the results, given that you know how to narrow down the results.
So, maybe give other search engines another try, while keeping that in mind. I personally very much prefer other search engines, because of that, even if I sometimes have to type one word more.
When I'm doing an online search and the top results are from quora I know I'll be disappointed. The result will be on topic but the information will be the same kind of one person's faulty rambling kind of thing seen in this post here. There is somewhat good content on Quora, it varies, but it seems to be way more miss than hits. Information from Quora is usually about as good as a hit from some old PHPBulletin forum.
I tend to respond obnoxiously to comments that are themselves obnoxious. If you check my history on both Quora and HN, you'd see that I am much more active on HN than I am on Quora. I tend to use whatever medium gets the job done. Best tool for the job and all. When you actually start submitting things at all to HN, send me a message so that I can learn from the best as to what the "proper way" to do things are.
The post got a lot of views, so I would call that a success. Not sure what point you were trying to make there.
Yahoo tried to compete with Google. Yahoo was first with "vertical search", about fifty special purpose question-answerers for weather, sports, celebrities, etc. Then Google copied that. Yahoo switched to reselling Bing, and gradually became irrelevant.
Cuil tried to compete with Google. Their idea was that their system was an order of magnitude cheaper to run than Google's. But they never developed a revenue model. Cuil went bust.
I tried a system which trimmed most of the ads out of Google result pages. That got very little use.
Google does have search customization. It learns from your queries what you ask about. At least if you're logged in, and possibly if you keep the same IP address.
Google is already moving past search to question-answering. They're not very good at that yet, but they're getting better.
I believe not; there is no need unless their ads become too intrusive. When we have issues with a technical issue in China, I start the VPN and search with Google. What takes my Chinese colleagues a lot of time, is immediately found in Google, even for vague components. Specsheets they did not receive are easily found on Google but not on the engines they have. It is not that we cannot think up a better Google, but I believe that means better NLP, not a better backend. Google would destroy anything that comes close, simply because it will be something they can bolt on in no time and then they have the data, money and infra.
The Language Council of Sweden was adding the word “ogooglebar” to the Swedish dictionary during 2012/2013 with Google with great force and success stopped that from happening.
When searching for answers which Google has no answer for the answer should be “this is ogooglebar”. So you don’t have to spend hours trying to refactoring your query for hours. This would be a great improvement and a great user experience.
This is Google’s akilles heel - never able to admit that they don’t know.
There is nothing that can’t be “beaten” through research and technology.
I mean, someday, we will be encasing entire stars in artificial shells and creating black holes of our own. What’s a single corporate entity that managed to get a lead in barely the first stages of a relatively new industry?*
How about incorporating search tech into web servers themselves? Move it to the level of DNS And other core protocols. Make individual servers automatically index all content that passes through them. They could then populate special caching servers with the indexes and results. When a user searches for something, the query gets sent to successive search-cache servers, kinda like a DNS lookup, until match is found or up to a browser-specified depth.
—
* People generally did not care much about search engines at the beginning; they usually just had a small list of sites in their memory or bookmarks that they visited regularly, and found others through magazines, links or word-of-mouth. Search engines were merely a hit-or-miss convenience, often used for porn or “wares” I’d wager, then that convenience emerged to become the core way of discovering and interacting with the internet.
Does anyone here remember the point when they “switched” to using Google? I personally can’t recall myself ever doing that. I think it just happened casually, through a browser that defaulted to Google. I remember thinking how ugly Google’s UI was, and preferred other engines for their “looks.” But at someone point, Googling just became faster than any other way to get to what I felt like looking up.
For now, I do wish browsers would start offering Wikipedia as an option for the default search. :)
I don't think Google search will be beaten. I think the whole current concept of search by going to specific URL/app, seeing a list of search result will become obsolete. We are already seeing that with voice based UI where you want an answer not a list of pages. The second part will be that lot of stuff that you care about will be known by google/Alexa and will be readily available so no need of searching. This also is already happening with google launcher on android where things like news and score about my favourite teams, travel times to airport or workplaces being shown nearing the time of travel along with routes so I don't have to search for any of these. This can be further integrated with a cab being available just when you need to leave. All of these earlier would have involved some kind of searches.
iMho this will be the natural progression. So if you are creating a better google search please stop you will be out of business before you start. This is no longer about standalone search but a larger ecosystem around the information needs.
I've been thinking for a while that a conversational search engine is the next iteration of search. If you query a human expert, you don't expect the correct answer immediately - it comes by filtering out possible answers with subsequent clarifying queries. The next gen search engine should be this way.
I want to know:
"In Rob Fergus' paper intrinsic motivation and self-play, how is it that the reward for agent A can be based on the time taken for agent B's task, when agent A and B are run sequentially, and agent B runs after agent A?"
Can Google answer this? No. Is NLP smart enough to answer this? Probably not. But the question itself is hard to write in one sentence. Complex queries are best structured as conversations, with clarifying back and forth, Socratic method style, like if you're asking a professor a question about hard science.
I switched to DuckDuckGo - it isn’t as good as google, but it is 90% of what I need. For the rest I have to slightly tweak my search criteria - but that is not a bad thing because it forces me to use my little gray cells.
YouTube and Gmail substitutes and then I am Google free.
Have you looked at protonmail and Vimeo? Both have somewhat of a free offering, although the main product is a few bucks. You know what they say about that though, you being the product and all.
For maps, check out openstreetmaps. Not quite as good as google, buy pretty solid. And you can run your own instance.
Heck, you could also run your own email without much hassle.
Google will inevitably become a blundering bureaucracy like every behemoth before it. The inevitable corporate politics will cause it to crumble from within. My bet is Chinese companies will outmaneuver Google outside western markets (US and EU). Also, Google is primarily an ad display company. Global marketing firms should support competitors to blunt Google’s dominance, especially outside the US. Finally, the EU will declare Google a monopoly. All this will take a long time though, just like Microsoft.
A better search could be made and likely is out there already, but one sums it up best. How often do your peers use Yahoo? Still, the company gets enough hits to be worth billions. Part of that is a global market, Yahoo is big in Japan supposedly, but part of it is your Uncle having made his decision pre-dotcom crash.
1. The post assumes that the search result quality is not good. This simply is not the case. Users say the search results are good. Could they be better? Sure. Are they terrible? No.
2. The post assumes users want transparency on the search algorithm. This clearly is not the case. The whole reason that google became what it is today is because of the lack of clutter and good search results.
If you remember Altavista then you remember users constantly having to fidget with all these knobs in the advanced section to get relevant search results. Lycos, excite, etc weren't much better. I remember using Google for the first time and it just worked.
3. The post assumes that users want deep customization. This again is not the case. Users want less clutter, less knobs, and a faster search experience. Google is delivering that.
4. The post assumes that any startup with a better algorithm can beat google. However this is not the case. Google has deep search integration with mobile providers such as it's own Android OS and also iOS. it also has it's own browser (Chrome) and has paid deals with Mozilla. Obviously, both browsers default to Google when you type something in the search field.
Not to mention, the leading IOT devices all connect to Google to find results when it needs to search the web. And for safe measure, Google has come out with its own voice assistant device, Google home, to safe guard against any shenanigans from Apple's Siri and Amazon's Alexa that might happen in the future. It is heavily subsidizing the price to make sure it has a strong horse in the voice game.
Integration aside, it also has scale. It crawls most of the internet several times a day. That's no small feat and it is quite hard for a startup to just replicate that with funding. I mean, even that won't cut it.
Google is now even using machine learning to understand relevant pieces of information on a webpage so when you query it by typing on your keyboard, by voice, or future mind control it delivers the answer you were looking for.
The only valid point the post makes is that Google has squashed competition by buying any startup that remotely could threaten it's position...for safe measure. I'll give the author that.