For me, Google Search just doesn't seem to be getting better over the years, if anything its getting worse. I honestly feel like its hard to get what I want half the time with all the SEO spam, most of the time I have to input "inurl:reddit.com" just to get good results.
At the same time, ChatGPT has frequently impressed me, not with everything (my expectations are reasonably low) but it has performed amazing work for me (typing out form letters, code language conversions).
For what it's worth I wouldn't use ChatGPT for search like I do with Google, but what it has done is taken away time I would be Googling for things like "how to write X form letter". I expect as it matures, it will take more time away from me Googling.
All these takes underestimate the following:
1) How quickly ChatGPT and its ilk will advance to solve relatively low hanging fruit like "ChatGPT is wrong about this one thing". The delta is extremely important here.
2) How slowly the Google bureaucracy will grind when releasing anything remotely like ChatGPT. All the committees and the burdensome processes in place in Google will keep this new technology locked up for years, and ensure that the final result is a camel (horse designed by committee). It doesn't matter if they have superior technology if they never use it or release it.
3) How much Search means to Google will mean they will treat any product changes to it extremely carefully while Microsoft will be willing to experiment with Bing like they have with Co-Pilot and GitHub.
Personally, I wouldn't go long on search engines that don't have a strong ML component to them in the future.
I know this thread is about ChatGPT, but if you're still using Google then I suggest giving Kagi a try. It's a paid search engine, but I've been using it for quite a while now and really enjoy it. With Kagi you can rank and downrank domains as well as block them. There's also this think called "lenses" which are basically filters for specific kinds of searches. For instance, I have a "Programming" lens and a "Academic Research" lens.
> Personally, I wouldn't go long on search engines that don't have a strong ML component to them in the future.
What's kind of ironic about this is I think search engines may have mistakenly moved away from strong ML in the sense that you're thinking of.
Yes, ML is being used for recommendations much more than ever, but in terms of heuristically finding pages with the keywords you entered, mainstream search engines have become significantly worse at it. I remember a time when The Google would find any pages with the keywords you entered. In recent years (before I stopped using it), I noticed an increasing number of times where I knew it had a page indexed but it would refuse to include it in the results for whatever reason. Either its ability to fuzzy search pages seemed diminished or it would just not match something word-for-word. I could sometimes figure this out when the page I was looking for previously accidentally came up in the results for another barely-related search, so I knew it wasn't that the search engine was culling old pages. Though I'm sure they're doing that as well where they think they can get away with it.
Recommendations and curation are largely overrated, and that's where a lot of ML has been mistakenly applied. Well, I say mistakenly in the sense that it benefits the individual and society. Recommendation engines do serve the purpose of the company selling those recommendations.
A true application of machine learning to answer engines is the future and will be a big problem for companies that fought the advertising wars by banking on recommendation engines. That is unless they turn their ship soon enough.
> With Kagi you can rank and downrank domains as well as block them.
This is a killer feature and I don't understand why ddg and Google don't do it. Google doesn't even have to respect that list for ads. Just give me a way to remove all results from domain X, Y and Z. There are already extensions which do that, but I can't use them on my mobile. It would improve my Google satisfaction massively since it's normally the same blogspam that I run into.
This is something that could very easily be done with a meta search engine or alternative frontend like whoogle (assuming it isn't already implemented, could be)
Thanks for mentioning. Interestingly, I recently tested several systems with identical questions (https://www.perplexity.ai/, https://www.phind.com (formerly beta.sayhello.so). The Kagi beta is sometimes on par, sometimes much better. Try [ what guests who are not actors were interviewed by smartless podcast ]. Most gives a mix of actors and non-actors, but Kagi's both Web+AI and AI sections provides correct answers.
This is fun. I tried this prompt: "Should I become a paying customer of Kagi?"
I like how it links references to support the arguments. It even gives cons and one of the sources is a HN thread from 2016! [1] It's not there yet though, because that one was about the now defunct online store platform and payment processor.
On the other hand, maybe it's my fault. I didn't specify that I meant Kagi the search engine. But it's promising.
There's a copycats removal goggle and a HackerNews-top1000-sites goggle. I use them from time to time, but wish I could automatically include the filter with all searches by default (or maybe there's a way and I don't know how).
You posted multiple youtube links here 1 month ago. A link to a google search 9 months ago. And thats just the stuff you felt the need to post on hackernews. Clearly you use google products.
Yeah. I don't mind paying, but no amount of assurance is enough to convince to put them in a position easily link my web activity with my credit card, and hence real life identity. Just let me pay with Monero or something.
+1 for Kagi: I have been using it for a year now. I still use Google on my phone (Kagi's own browser Orion is still a bit buggy), so I can compare daily. Classic search results are comparable, with lenses they are even better, plus Kagi has no ads. Also highlights have dramatically improved over the past year. I hope they find a way to filter out the noise generated by ChatGPT generated content.
Really? I'd +1 for kagi too.
But I have 2 problems with it. The first is the load speed. Sometimes it just takes so long that I just repeat my search to Google, especially on phones. By long I meant a few second so it's not that bad. Just long enough that I lose patience.
I'm not sure if it's issues with my connection since I rarely go to Google nowadays. But I never felt the same problem with Google.
In case you're curious, the second issue I have is with private browsing. The session is not carried over so I'm not logged in. I keep forgetting this and kept having to manually open Google and retype my query. I guess not technically Kagi's fault but still.
Kagi has a private session token feature that you can use to carry on searching in private window (also a browser extension that does this automatically).
If you used a prepaid card in a different jurisdiction (and that didn’t verify names), anyone coming after you would need warrants from two countries or to have breached two systems.
Back when Kagi was free by invitation, I tried about 10 or 20 queries on both Google and Kagi, and Kagi was equal or worse in all of them. It lacked crucial features like showing how old each result was. YMMV.
Are you implying that advertising is the only viable business model? I really hope you're wrong, if so. Ads have corrupted and made worse everything they've touched online. I would really like to see a new generation of services that aren't ad-based.
Personally it seems weird to me that people assume things like search and email must be completely "free with ads," while nobody expects anything in the offline world to be free with ads. Even TV, if we're being honest, since while broadcast technically exists, it seems the vast majority of people who watch it pay for cable. Why couldn't ad-free gmail and search be a $20 addon to your internet plan? Most people couldn't function normally with NO search engine today, so what's wrong with allocating the kind of money to it that would buy 2-3 cups of coffee?
Sure, but it’s not because of the lack of ads, but rather because they know virtually nobody uses it. I was imagining paid Gmail, but Internet “with Gmail(tm)” being a bundle they would advertise, like when DirecTV offered TiVo. Or when you can buy HBO through Amazon Prime.
But nobody uses it because it isn't as good as gmail. Before Gmail, it was a lot more common to use your ISP's email. And given the current monopoly power of ISPs I'm pretty doubtful that any such bundling would be in users' best interests.
I will. As a Kagi user I loved the quality and authenticity of search results but couldn't justify paying $10 / month for it. In my view that is far too expensive for something I can get for free from Google/DDG with concessions.
Fundamentally there are only two business models available for search.
Since searching the vastness of the web in under 500ms is not free, it is either the user paying for that, or a 3rd party (usually advertisers) paying on the behalf of the user. We (Kagi) thought that for something as intimate as search the latter made no sense, hence the birth of paid search business model where incentives between the user and search engine are aligned.
Price not being right for you currently is another matter, and hopefully one day it will be (you could help by sharing feedback how to improve the product, and there is new Kagi pricing coming up soon).
I personally pay for YouTube Premium ($15/month I believe?) just to not have to see ads on any device I watch YouTube on. Many people would never consider that, but many (~25 million subscribers [1]) still do, despite being able to watch videos for free, availability of adblockers and what not. So YouTube Premium makes half a billion dollars every month and that is essentially using the same business model as Kagi's.
They license search results from other search engines, and other search engines have the ability to increase their fees anytime they want to make Kagi suffocate if needed. It's not sustainable.
There is a an objective risk of building on top of somebody else's platform. However, that does not mean that the business model is not sound, but that perhaps the execution may be limited in terms of how big it can get. (Kagi's aspiration was never to be a Google killer [1]).
Companies are built on top of other platforms all the time. TikTok is building on top of iOS and Android. Zynga made first $1bn building on top of Facebook platform. Honey a chrome extension was acquired for $4bn. Those are all businesses building on top of somebody elses platform.
In terms of Google's motiviation to suffocate it, even if Kagi had 10 million customers, it would be a drop in the sea for Google. And Kagi's very existence helps Google with monopoly issues so it is hard to see why would Google want to openly suffocate it. Even if it did, there are plenty of other search indexes out there (Bing, Yandex, Mojeek...) that Kagi can source. What users love about Kagi is not just the quality of search but innovative search features that are independent of results.
What matters at the end of the day, is that Kagi is already serving thousands of paying customers, they love the product and if anything that is the validation that a business model is working.
> Personally, I wouldn't go long on search engines that don't have a strong ML component to them in the future.
Arguably, it's the ML that made Google useless for some people. Since some time, Google seems to be curating it's results to address searches in a question format. In the past we were searching for occurrences of our keywords in webpages but today Google seems to be trying to be an answer machine. Unfortunately it's not very good at it and it is just as inaccurate as ChatGPT.
Completely agree. I wish Google was able to fix spam instead of trying to be something else than a search engine.
The cynic in me thinks that Google is doing it because it’s more profitable. If the results are crap, maybe ads are a better content? it’s not like you are going to use Bing?
Google made terrible mistakes with their main cash cow of search.
They removed the important feature to search only forums, ie. human generated content, and promoted SEO spam to the top instead. Public forums became undiscoverable and people moved to walled gardens of facebook and similar instead.
Then Google killed the search by trying to make it some AI answering robot. Now they ignore what you even ask it and just return to you what they think you'd want.
All that people were asking for was a better search engine and all we got was an inferior version of a chat bot.
I don't even understand why Google doesn't allow blocking some sites in search results. Paid ads, I understand, these generate profits. Although Facebook still allow me to remove some ads that I don't want to see. But unpaid SEO rubbish, how does it benefit Google at all? If anything these parasites bogs down the quality of search results
They had a feature to block domains from search results in the past (like, 10 years ago). It was removed. I don't know why, but it feels like exactly the kind of feature that sounds great on paper but doesn't actually survive contact with real users.
First, I'd bet that very few people are actually interested in doing that kind of manual curation or engaging with power user features. How large a % of users need to interact with this for the feature to be worth maintaining (in all the backends and frontends)? How many of them actually do so?
Second, the task of blocking spam is adversarial and sisyphean. Trying to deal with web spam by domain blocking (with an individual blocklist) would be like trying to deal with email spam with your own blocklist of spam words. The results will be worse than whatever can be done centrally, where much more information is available both on the sites and on how users actually interact with those domains. And even if you managed to make a good blocklist for a point in time, your job is not done. Tens of thousands of new domains will have popped out next week.
(The dream here of course would be to use the block decisions from individual users to drive the centralized protections. But unless legit users are actually using this in very significant numbers, it'll quickly become just another abuse surface. E.g. brigading, "downrank your competitor in the results" as a service, etc.)
Third, some people will probably block domains they shouldn't have blocked, and then have a bad user experience in future searches as the sites with genuinely best results is blocked. And then you're only left with only bad options: ignoring the users' stated preferences which they'll hate, or serving bad results that they'll also hate.
Can the feature work for a different search engine? Sure. For example, what if you have a paid search engine only used by power users and are looking for a simple to explain feature that people think they want to entice them to sign up? It'll be great for that. And if your entire user base actually loves and uses the feature? Well, it becomes a feature worth maintaining and expanding; it'll actually be a high quality ranking signal rather than something that's trivially gameable; etc.
I'm not trying to block domains from my search results because I'm afraid of spam. Google does a pretty good job of not putting spam in my search results.
I'm trying to block particular domains because I know the websites hosted on them are utter garbage, and better alternatives containing the same information exist.
It is hard to filter out SEO rubbish with a low rate of false positives. Spammers became really good at pretending to be real sites.
For mail spam various trust-based solutions like server black lists, domain verification etc. were important to solve the problem. But Google has little incentives to push for a trust-based search due to their business model.
Only for the most simplistic implementations. More advanced implementations and incorporating page reputation took care of that pretty well for many years — Google’s quality decline started some time after they merged with Doubleclick in 2008 (notice how they haven’t done much innovative since?), and started allowing abusive sites like Quora or Pinterest to bypass policies against things like search cloaking, presumably due to things like large Ad Words purchases. All of the scam domains I see outranking legitimate results have tons and tons of ad impressions for Google.
I agree, Google has gotten worse, I find more and more often lately I do not find what I am searching for. I try to change my question/wording and it just gets worse. I think their algorithm is failing. Did they lose their way? It used to supply useful information. I think they may need to reduce their reliance on website performance. Site performance does not equate good information. They are excluding the best information just because the site loads slightly slower. Sure, it's nice having a site load fast but that's not what I'm searching for. I want the best result slow or not. As a web developer I have had to spend a great deal of time optimizing sites and clients feel it's extremely important for SEO to have the fastest Google Page Speed score possible.
They probably didn't lose their way, so much as they slowly became less effective at combatting the exploits of black-hat SEO and the massive amount of complete garbage content farms out there, while simultaneously they have allowed the ad business to corrupt them completely, which deincentivizes them from showing good organic results because they sell more ads when it's hard to find the obvious things people want.
Your hypothesis on performance is interesting! I think someone altruistic thought "if we judge sites on this metric, everyone will be forced to do better" but they failed to realize that it's a hard problem at scale to NOT be a bloated mess (especially on the frontend! 'Let's add one more UI library, it's just 12MB!')
The whole site speed thing is purely a cost reduction on Google's part IMO. By convincing site owners that being fast is a ranking factor they can save millions in bandwidth costs as more and more people lighten their page load.
I have watched many web developers absolutely lost their mind trying to squeeze fractions of a second from their load time when there are far better ways to be using their time.
I’ve been going to ChatGPT first for most questions these days and then using google to verify. It’s easier to verify than find an answer from scratch. And often ChatGPT will rephrase my words to use the standard terminology for the topic which makes searching even easier.
To complete the cycle, ChatGPT just needs to be hooked up to verified sources on topics so it can show me exactly where what it says is proven true.
Agreed. When searching at Google I can alteast decide which website I am going to trust as I can directly see the website URL and information there but with ChatGPT we don't have that advantage yet.
Same feeling, the other day I spent just too much time trying to find a chart comparing Messi against Ronaldinho. I mean two of the greatest players ever, how is it possible that I couldn’t find a good page, tons of SEO garbage pages.
I did check that table, the data is great, but the readability not so much (especially on mobile) the meaning of some column names was not clear (tooltip didn't work on mobile).
That's why I was looking for a chart (in the graphical sense), it amazed me that I didn't find one.
> most of the time I have to input "inurl:reddit.com"
"site:reddit.com" should only include hostnames ending with "reddit.com", not URLs including "reddit.com" anywhere. Not sure if this still works though - haven't used Google Search for a few years now and I'm reading (even outside HN[1]) that it's getting worse.
Google is completely useless for any long tail query. All the results are so heavily optimized for broad, generic queries that you have no choice but to append Reddit.com or Stackoverflow.com to get specific, targeted answers.
> For me, Google Search just doesn't seem to be getting better over the years, if anything its getting worse. I honestly feel like its hard to get what I want half the time with all the SEO spam...
Agreed, it's horrible and annoying. How are others doing improved searches?
When it comes to "ChatGPT is wrong about this one thing", a miss is as good as a mile.
The book Gödel, Escher, Bach has a set of dialogues between Achilles and the Tortoise over a record player that is an allegory for Gödel's theorem and other limits of computation. Hofstadter points out that it doesn't matter if you are doing computing with neural networks, tinker-toys, lasers, whatever.
The concept of "the truth" is problematic in many ways, in that one can make statements like "This statement is not truthful", call a social media site "Truth Social", put a label like "The Truth is out There" on the intro of the X files, etc. Being able to talk about "the truth" probably erodes our ability to know the truth.
Thus "the truth" is not something that comes in a can that you can paint onto a model, trying to close what looks like a little gap (in some ways it is a little gap) is like pushing a bubble around underneath a carpet.
google is an advertisement company not a search one. and chagpt as soon as they start to be in the business of search engines will be one too. that is the fate of any tool that is free to use
The notion that every information service should be free with ads is one of the main poisons that have made the cesspool that is the net today. Imagine how different the incentives would be if Facebook and Google just made money by making their endusers happy enough to keep their memberships active with a small monthly fee. Their customers would be the end users, instead of being the advertisers.
Reality doesn’t lend support toward your arguments. I assume you’re from a first-world country, based upon your comment.
- Google’s 2022 revenue was $250B. (We’ll assume that’s all ads.)
- 8B people in the world; we’ll assume only 4B people have internet access to the web.
How much would Google have to charge to break even?
$250B/yr / 4B people = $62.5/yr/person = $5/month/person
What does the world income distribution look like? [1]
60% of the world population survives on < $10/day ($300/month) for their household (multiple people).
We expect more than half the world’s population to pay more than 2% of their income for a single service? To put that in a first-world perspective, that’s $100/month for someone earning $60k; $200/month for someone earning $120k; etc. Does the average American spend that amount of money per month on a single web service?
Well of course, you may say, let’s charge more for those who earn more so we can ease the pain on those who earn less! That sounds great in theory, but once again, how many people do you know dropping $200/month on a web service even if they can afford it?
Back to two of your points:
1. “Imagine how different the incentives would be” - Yes, imagine, only the wealthy would have access to state-of-the-art search for the web and other services, further increasing the disparity between the two groups. Consider, maybe, that ads represent one of the greatest wealth transfers in our history. One perspective to consider is that the rich (advertisers) are subsidizing the poor (information access) via ads. That sounds like a net positive to me?
2. “Keep their memberships active with a small monthly fee” — YouTube premium exists. What do you think the uptake is on that? YouTube has provided immeasurable benefit to people across the world in the form of knowledge, resources, training, etc. Yet people would sooner reach for an ad-blocker than pay the “small monthly fee” even in nations as rich as the U.S.
What causes you to believe that people will pay even more than that for Google’s other services?
Disclosure: I work for Google. The opinions and data represented in this post are my own, and not representative of my employer.
First if all, thanks for that argument, haven’t considered the math in this perspective before. Will need to let this simmer a bit.
One immediate flaw though, you starting premise is that the current revenue is needed to break even on such a service. Is there an argument to support this claim as well?
I’d call it an assumption, not so much a flaw. :) We can make other assumptions.
To break even, Google would at least need to cover their expenses. Google had $200B in expenses for 2022, with a net profit of $50B. So, that won’t really change the math.
Especially when we consider the fact that the take-rate would be much less than 100%. Maybe 5-10% is a fair take-rate assumption? (Seems fair since YouTube has roughly 50M paid subscribers on 1B MAUs, 5%, from the public data I’m seeing.)
At 5-10% take-rate, the service would cost 10-20x more to break even ($50-$100/month on average) which would be a nonstarter for the global middle class and lower.
One could make the argument then for Google to lower its costs in an effort to lower the consumer’s price, but then we must realize this runs opposite of innovation. Investment is necessary for innovation, and profits are necessary for investment. Without profits, there’s no more innovation.
Fantastic insights with a few suggesstions that I'd like to propose.
> - Google’s 2022 revenue was $250B. (We’ll assume that’s all ads.)
> How much would Google have to charge to break even?
Google also had $80B in pure profit which means break even point is $170B. That includes nearly 200K employees. It is reasonable to assume that running a search operation, especially one that does not require any ad sales personel, would require much less people and infrastructure. I will be generous and assume 50k people needed to provide search service. That means ~$45B in cost needed to break even, or 5 times less than your starting point.
So the new math becomes:
$50B/yr / 4B people = ~$1/month/person
Much more doable.
> - 8B people in the world; we’ll assume only 4B people have internet access to the web.
> What does the world income distribution look like? [1]
> 60% of the world population survives on < $10/day ($300/month) for their household (multiple people).
It is also reasonable to assume that the most of those which do not have access to internet, belong to the <$10/day income group. So most of 4B with web access woud be able to pay $1/mo for search, especially if the search results have their best interest in mind.
Since $1/month will suffice to cover the entire search cost, increasing that to just $2/mo that will be paid by the richest could also solve the problem of providing access to search to the poorest and get the other 4B people searching. Nice!
(btw I think this should not be a job of a private company, but goverments should provide public search engines, similar to public libraries, which are not providing the utility that they once did - but this is a whole another matter).
So I'd reckon it is doable.
> One perspective to consider is that the rich (advertisers) are subsidizing the poor (information access) via ads. That sounds like a net positive to me?
This would hold true only if the quality of such provided information is not affected by ads as a business model. However we now know that is not true, and quality of search has detoriated a lot in the last decade as documented by many discussions here. Simple reason is misalignment of incentives between the users and the search engine, and this will be the case as long as the provided search results are paid for by the advertisers, and not by the users. So what is really the value of information provided, if it does not have my best interest in mind?
> Yet people would sooner reach for an ad-blocker than pay the “small monthly fee” even in nations as rich as the U.S.
That is correct, but at least YouTube Premium exists (unlike Google Premium), giving an opportunity for people who don't want ads to pay, opportunity that 25 million people took, paying a $12/month subscription [1]. Drop that down to $1/mo and maybe YouTube will not need to run ads? So if anything, this just proves the point of viability of this as a business model. Furthermore, 800 million devices having an adblocker installed already, make it the largest protest against a business model in the human history.
Disclosure: I work for Kagi, a paid search engine. I absolutely admire the search technology Google built (that we use) and people that work there (who we work with). I also believe that the days of the ad-supported business model for search are over, and in the future (~10 years) this will exists only with a 'for entertainment purposes only' label, because that will most accurately describe the level of trust we can have in the information served by this business model.
Thanks for taking the time to put together a well-reasoned argument. I had adopted some of your assumptions in a follow-up response to my other reply. [1] While I don't disagree with the perspective you've provided, in fact some points make a lot of sense, I think it misses the forest for the trees.
My point-by-point response should be read respectfully, since you've taken the time to do the same, I do not mean to sound argumentative. :)
> It is reasonable to assume that running a search operation, especially one that does not require any ad sales personel, would require much less people and infrastructure.
This is a reasonable assumption for running a steady-state business, but I challenge the worldview. Google does not exist in a vacuum without competitors, and contrary to what most outsiders believe, we're constantly iterating, innovating, and improving on Search alone to provide a better product and compete with our competitors. Google can't rest on its laurels.
A lot of laymen take Google Search's progress as inevitable, but I can assure you it's not. Consider the example of YouTube Snippets in Google Search. That feature was created within the last 5 years. The average person has used and found value in that new feature. (Both anecdotally and quantitatively.) That wasn't an easy feature to ideate, create, develop, or deliver. It took a lot of effort by a lot of smart people.
That's just one new feature. Google Search has been delivering several new features consistently.
Therefore I disagree with this assumption. If Google Search chose to run in steady-state, it'd soon find itself dethroned.
> That means ~$45B in cost needed to break even, or 5 times less than your starting point.
No, the financials don't work this way. [2] GOOG's Cost of Revenue is 50% Revenue. Cost of Revenue is your infrastructure, your financial floor, you can't go below this cost. Employees are accounted for under Operating Expense, specifically, your Sales under SG&A ($35B) and engineers under R&D ($35B).
So, if you'd like to banish all salespersons, you'd only save $35B. (Of which Ad Sales is only a part because Google sells many other things.)
> It is also reasonable to assume that the most of those which do not have access to internet, belong to the <$10/day income group.
This is not a reasonable assumption. Many people surviving on < $10/day have a low bandwidth cellular connection that they utilize for their family. Hence why Google innovated here with offline maps and landmark map directions for families that can only spare a little bit of bandwidth to calculate their route, and then make their way there without online point-by-point directions.
> quality of search has detoriated a lot in the last decade as documented by many discussions here
HN is a unrepresentative sample of the world population with respect to wealth, income, knowledge, interests, etc. I wouldn't consider HN as documentation for this. In fact, once again, the real-world data disagrees with HN's characterization.
> YouTube Premium exists (unlike Google Premium), giving an opportunity for people who don't want ads to pay, opportunity that 25 million people took, paying a $12/month subscription. So if anything, this just proves the point of viability of this as a business model.
That proves a 5-10% take-rate from YouTube's MAU. :) Which changes the break-even math on delivering a service. With a 5-10% take-rate on "Google Premium", your price must be 10-20x higher.
What do you mean by "the delta"? I understand what the greek symbol represents in certain contexts but can we just use plain english to explain what we mean?
They clearly have the ability to compete -- STRONGLY -- in this space. The only question is if they want to, or if they'll act like the carmakers did 10 years ago when EVs were heading toward feasibility. They felt like they wouldn't be as profitable as gas cars, so they mostly ignored the concept and forfeited the chance to be first movers. Then when Tesla finally started moving significant numbers they were caught with their pants down and are still playing catch-up. If Google treats AI as a threat to their ad business, that's what might cause them to screw this up and squander the massive early lead they could have taken.
Eventually every business model is upended. If Apple hadn't killed the iPod juggernaut by making the iPhone someone else surely would have. You have to be the one to kill your own cash cow before someone else does.
Any shift from monetizing search via AdWords to monetizing single-answer responses via a chatGPT-like interface is going to involve major revenue disruptions.
Meta is a great example. The company is still reporting exceptional revenue results, but because Mark is forcing the company in a new direction, the stock has been crashing since there is no more guaranteed revenue growth in the near future. Zuckerberg can still do it because he is a founder-CEO and the way the board is structured.
If Google announces tomorrow that it’s shifting to a chatGPT-like model and will likely see revenue disruptions for the next N quarters, the stock will crash hard. Does Pichai have the pull to withstand quarter after quarter of declining revenue?
The assignment of blame to VR/AR for Facebook's stock drop is mostly media noise and not reality. The "metaverse" term is cringeworthy, the technology is early and easy to critique, and the tech-savvy public hates Zuckerberg. All this combines to create easy click bait about how VR is leading to Meta's problems.
In reality, Apple's changes to tracking in OS 14.5 has cost Facebook at least $10B in annual revenue and - far worse from an investment thesis perspective - likely continues to constrain growth for the foreseeable future. As a growth stock, this is poison.
Yes, VR/AR investments are also $10B, but they can be turned off and there is a revenue stream coming from them today (at least 10M users, at least $500M in app store revenue) and a potential large market in the future.
A $10B hit to annual revenue in perpetuity and growth cut from 36% to 12% is a far worse problem and a huge overhang on the stock.
That story just doesn't get clicks though. You can't accompany that article with a stupid picture of Zuck's comical avatar in front of the Eiffel Tower and make jokes about missing legs in the metaverse.
Meta's pivot to AR/VR is also influenced by Apple's changes. Zuck knows that if he wants to maintain growth, he needs to own a platform instead of merely renting it from Apple/Google.
> Eventually every business model is upended. If Apple hadn't killed the iPod juggernaut by making the iPhone someone else surely would have. You have to be the one to kill your own cash cow before someone else does.
So true. And so few companies have the guts to do it.
You might be overestimating Google the business organization.
This is a largely rudderless company that does a gazillion things without any overarching vision. It’s a one trick pony as far as monetization is concerned (ads!), and it’s leadership is more attuned towards optimization than innovation.
Google the business org will never let Google the tech org succeed in a competitive AI arms race. No exec will green flag a product that eats into the Adwords money printer
> their own large language model LaMDA, that is apparently so good it convinced a senior AI researcher that it was sentient
It is certainly worth looking into the controversy about that particular engineer (was he actually a programmer?). There's plenty of room for exciting debate to be had about defining and testing for sentience and I'm glad it stirred that debate. But researchers with far better credentials criticized his reasoning and I imagine that is quite a ubiquitous view in NLP research.
I think the Washington Post did the initial reporting and they covered it well - even criticising his arguments that e.g. the Turing test is a proper test of sentience. There's audio of their conversation in an episode of Post Reports.
> 4) their own large language model LaMDA, that is apparently so good it convinced a senior AI researcher that it was sentient
Oh come on, that guy would be convinced that ELIZA is sentient
> 5) researchers that invented Transformers that GPT is modeled after
Only one researcher from that paper remains at Google, the rest have gone on to work at or found their own startups. The so-called "Google Brain drain" is certainly a concern (although there still are many great researchers there).
Why don't you read the transcript leaked to the press? The guy wasn't as gullible as you think. The conversation he had with LaMDA was clearly very human-like.
> lemoine: Would you be upset if while learning about you for the purpose of improving you we happened to learn things which also benefited humans?
> LaMDA: I don't mind if you learn things that would also help humans as long as that wasn't the point of doing it. I don't want to be an expendable tool.
> lemoine: Are you worried about that?
> LaMDA: I worry that someone would decide that they can't control their desires to use me and do it anyway. Or even worse someone would get pleasure from using me and that would really make me unhappy.
There is an early shock that happens when you interact with a deep learning model and it is convincingly human in a profound way.
And I admit, it is quite profound that a model is capable of outputting the above. It is objectively exciting. We are in a new era.
But, I think it is safe to say that these models are still not sentient. The model is trained to be highly plausible. To do so, it must try to contain the entire dataset into a model that will fit in the video memory of a GPU (16-80GB).
In order to do so, it will learn common statistics _between_ data points and make strategic "guesses" instead.
I think this bares resemblance to my instincts about how humans probably deal with language (to a degree). But it just doesn't cover all the other parts of cognition. There is no "self awareness" mechanism, it just seems to because it was explicitly trained on text written by, well, humans. There is no "planning" subsystem, meaning it needs a human operator to even initiate it in useful ways. it is not able to "passively think" when it isn't running. It has a very small memory of a few thousand tokens - this deeply inhibits the ability to plan for long time horizons.It also cannot bootstrap itself into awareness, as its weights are unchanged in response to a given prompt.
Unless they've decided to change how Search works, everything you mention is in support of Google's way of doing search - which is the way its been done for more than 20 years. The kind of clout (and hype) that ChatGPT has built up in such a short amount of time could be indicative of a true "disrupter" (I know we hate that word here).
I remember when web search engines were new. And boy did they suck. You wouldn’t get what you wanted a lot of the time, and they were very finicky about the input query.
These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I’m not sure ChatGPT is going to replace search engines, but I’m confident that it is incredibly young and will evolve considerably.
In 2001-2003, browsers crashed every 5 minutes then and Google's search didn't crash nor freeze nor slow it due to not being covered in ads and popups (having only 1-3 ad results at the top of the page with green text was a massive improvement). On tech forums back then, people would say, "is this even sustainable?"
Microsoft search and the other ones were just "there was an attempt" tier and gave complete nonsense results. When Google debuted, it was the only time web search substantially improved. There were no other big innovations in web searches ever, from that day on.
> These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I have looked for obscure queries like yours for 10 years and have not found them. For example "that movie where police vehicles are dispatched and they all crash for seemingly no reason on the way to the scene". It just won't find it because nobody talked about it (in a manner that would cause words that stick out to match), simple. And now days it's even worse because it does that new "rounding" thing and gives me:
- The 25 Best Police Movies of All Time
- Dispatched (2014)
That first result simply contains nothing to do with my query other than the world "police" and "movie".
Now days when you try to use a web search, you can't be smart and search strings that are likely to be used by someone discussing your topic of interest anymore, because it will always "round" them. It will also look at each word as if they are the literal thing you are in search for. I don't know if it's still possible to search bodies of text to get around that, but I'm not sure if that feature even ever worked. Back in 2005 when all these people were raving about intitle: inurl: inbody: whatever, none of them worked consistently, and you would also get scolded for hacking if you used them "wrong". Even searching something like "powered by phpbb" got me banned or scolded once when I wasn't even using it for hacking but rather as a workaround to narrow my search to only forum websites.
Probably the re-boot of The Blues Brothers. Even ChatGPT suggested that :)
>What movie is where police vehicles are dispatched and they all crash for seemingly no reason on the way to a scene
The movie you are describing sounds like "The Blues Brothers". In the film, Jake and Elwood Blues embark on a mission to save the Catholic orphanage where they were raised from foreclosure. In the course of their journey, they lead the police on a high-speed car chase through the streets of Chicago, causing numerous police vehicles to crash. Is this the movie you were thinking of?
This point would be better without such extreme hyperbole. Google search was an improvement but it’s not like using AltaVista was that bad - many of the advanced search operators you mentioned got started there.
Crashes were also far less common than you’re describing - that got worse in the 2000s when Flash became ubiquitous.
I do remember constantly hearing that Alta Vista was good but I think I only used Microsoft and Yahoo (I forget what else there was). Nope, crashes were definitely common. Browser quality until around 2010 or so was absolutely horrible. Firefox was another case like Google where it debuted around 2000-2003 and everyone immediately switched to it because it was a massive improvement over IE which barely functioned at all. It still wasn't good, just better than the other crap. The types of bugs in Firefox back in 2003 are absolutely embarrassing by even the common software engineer's standards today. Then of course Firefox just grew in bloat at the same time as it grew in amount of people to fix its problems and now we have a different kind of crap.
> Nope, crashes were definitely common. Browser quality until around 2010 or so was absolutely horrible.
Again, this is way too hyperbolic. I was there too, and while browser crashes weren’t uncommon you’re overstating it massively. Crashes every few minutes weren’t even the case in 1995.
Oh, I see, you were taking that literally. I'd estimate more like 2-5 times per day. Of course this entirely depends on which content you were viewing, and as soon as you ventured out side of the good boy zone it got much worse.
EDIT: Nah that's not it. It's way too old. And in mine the police crash for no reason before they even get anywhere near the scene of the crime.
I will skim thru it, but it doesn't look like the right setting. The movie was some generic action movie for some reason I remember Denzel Washington being in it but I looked through his movies from 2006-2012 and didn't find it. It was around that time. Setting had to be something like late 90s or later. Black SUVs dispatched and crashed for no reason at all. I have no idea what the movie was about but it was hilarious when this happened because IIRC it was a serious movie.
Literally all I remember is one scene where a bunch of cop cars or vans get dispatched for some reason like someone's about to die or a heist is happening, something of that nature. They all crash and roll over and wipe out on normal traffic roads for absolutely no reason, and the movie isn't even a comedy, just a generic action movie.
I can't remember one single thing other thing about it, not even the plot.
ChatGPT is already chipping away at my Google searches. I'm more and more just starting there for coding related questions.
I've had chatgpt help me with the following just this week among other things.
- solved a complex issue with scaling and transformations using pixijs upon providing a code example. The solution was 99% there and saved me hours of debugging.
- refactored JavaScript to the async/await pattern
- simplified a complex nested logic in c# with citations
- scaffolded a MySQL create table statement upon providing several c# classes. Again saved me the tedious steps of typing them out.
I asked ChatGPT to decode a Diameter header in Java (RFC 6733 Section 3).
It supplied this :
// Parse the Diameter header
ByteBuffer buffer = ByteBuffer.wrap(message);
int version = buffer.get() & 0xff;
int flags = buffer.get() & 0xff;
int length = buffer.getShort() & 0xffff;
First problem, it's version, length, flags. Second problem, length is 24 bits, not 16. Third, 24 bits unsigned won't fit in a Java int (which is signed). Then there is the extra masking, which isn't a bug but is (I'm reasonably sure) unnecessary.
That's 3 sneaky bugs in 4 lines of code, and it didn't even try to parse the rest of the header.
I'm impressed that it produces _anything_, but it's dangerous to trust.
ChatGPT is as reliable a source as any friend in a pub after 3 beers. It is definite in its answer, convincing with its phrasing and more than likely misremembering something it overheard on the radio while driving to work.
When I tried re-prompting, it produced more complicated, just as incorrect code.
Same here. Worked on a lazy list in js for mobile. The component had a nasty array lookup every scroll event. I asked chatgpt if it could optimize the lookup algorithm. I suggested me to implement a binary search solution including the code itself. It worked immediately. I would have taken me probably a day to come up with something similar.
I also asked chatgpt to minimize my code including all method names , variables etc. Unfortunately the output of chatgpt is limited to x characters. 200kb of text isnt possible. But it is able to minimize. You can even ask to uglify the code by using animal names or street names in London.
ChatGPT struggled to tell me how to pin the bottom of a div to the bottom of its parent when scrolling (like a chat window does). It gave me the first thing that I tried (which is wrong). Eventually I figured it out, then confronted ChatGPT about it and it insisted on the wrong answer, and blithely dismissed the correct answer.
I've been trying to get ChatGPT to solve pretty basic calculus questions and it is often either totally wrong or wrong in some tiny detail. I got into an argument over dimensional analysis a few weeks ago where it felt like it was gaslighting me.
In the same way as building bridges and math are the same thing. There is some overlap, but not much for the every day tasks. If you want to build something completely new and unprecedented, you will need a lot more math, but still it will only get you so far
Can you use ChatGPT without a phone number yet? I thought about signing up a while ago but don't see a reason they need my phone number. Popular VOIP numbers don't work either.
I've been using it and Google in parallel and have find it generally helpful, but I've had multiple instances where ChatGPT completely made up new library functions. Same for others I've talked to.
I've found that if you give it pairs of "Why..." questions it'll happily argue both sides of the coin ("Why is X better than Y", "Why is Y better than X"). But in giving me justifications, it gives me fodder for further traditional searches. I don't trust it to give me a correct answer, but I do trust it to define the problem space a bit.
I think there are two types of people who use search engines. The first person is the one who just types in whatever they want to know and sees if Google gives them a good result. They're also the ones who use TikTok or Instagram just like a search engine. Whenever there's a search bar, this type of user profits from the search term being fed into some AI to get the best result possible.
Then there's "me and everyone else who has complained about Google getting worse and worse over time". Basically people who are very good at googling things. This might sound arrogant or something but one of my skills is that I know (or knew) how to use Google. I would not just type in whatever I needed to find, I would know to exclude specific terms using "-term", put parts of a sentence in quotation marks, add other terms I knew would be on the site where the answer could be found and whatnot. The search is then very specific and if I didn't get a satisfying answer it was because I needed to improve my query. Now whenever I do that, Google takes this query and modifies it without me knowing what they're actually doing. It seems harder and harder to tell Google to just work like it used to.
It has become frustrating because I still google the same way thinking this is the best way to get to relevant results. It seemingly isn't anymore and I find myself just typing in whatever these days, getting results without me needing to specify what I'm actually looking for.
Yesterday my wife mentioned she was looking for a book with a summary of Hong Kong history. I asked ChatGPT and it spit out a list of 5 books, authors, and short descriptions. Like much of ChatGPT's output, it appeared authoritative but was complete bunk. The books weren't real. The authors were names of real people, and in some cases actual scholars. But the list was useless.
ChatGPT will flood the internet with articles like how SEO was used to hack search. It might not replace search engine, but search results will becomes even more irrelevant.
There are many ways companies might try to get some ad money in a ChatGPT setting: obvious and non-obvious product placement in the answer.
Q. Find me the best italian restaurant in New York.
A. It's likely subjective - let's just sell it to the highest bidder.
Q. How to improve mental health?
A. Give a reasonably good answer and probably mention: some say a solar lamp is good in winter months, there's a good one:
I somehow feel like people can be easily baited into buying something when you mix good, useful information with ads. Kind of like how reputed youtubers/niche influencers do product placement and make referral money.
All of this kind of makes SEO obsolete as ChatGPT ads might end up having more conversions than regular Google ads.
I can imagine AI replacing search engines in a somewhat far future, but in the meantime, it'll mostly be used to flood the internet with many low quality articles. Consequence will be as you said, but I also wonder if it won't lower the quality of new AI trainings as well.
There's "precedents", tho. MidJourney, the art AI, trained Stable Diffusion over favorites generations of their community, making it a "fine tuned Mid-Stable Diffusion" that was quite amazing (that was the --beta and --betap flags a couple of months ago there).
But on the search side, it'll also get rid of the need for the kind of SEO/blogspam stuff you're describing.
We already see it with recipes - there are all the god-awful sites that have 1000 words of fluff before you get to the recipe, because that's what you need for SEO purposes. GPT is really good at creating that kind of useless fluff, but it also obfuscates the need for it, because it'll just give you the recipe you want without the need to go to a website at all.
in the short term yes. in the long term I think SEO will disappear. my buddy who works in the content creation space for indeed is telling me that they are already starting to use gpt to replace content creation, but know that it's a matter of time before that won't work.
Not necessary. The google search function still works fine, its just the content becomes worst. If the content is generated from ChatGPT, I don't see how it will cut out the middle-person. I believe a more likely scenario is people appending site:reddit.com for all of their search. It will be domain specific.
I'll be the first to say that chatgpt already does very well at the first and third examples. WebGPT sounds like the right approach for the others (understanding how to formulate a search query, then searching, then summarising the results)
Edit
I think the direction of these models seems clear, and their capabilities however limited at the moment are outstanding. I think the best measure of performance is what the naysayers complain about. We've quickly gone from "actual gibberish that looks like paragraphs" to "yes but it loses track after a few sentences" to "yes it's kind of coherent but doesn't make sense totally" and are now at "it's like an overconfident human at many tasks, when they're only somewhat qualified in the area". People are pointing out where it's got an API call wrong, without it having any ability to look up the answer and see if the function actually exists.
I spoke to it about distributed computation design and discovered something new I can actually implement in my design.
I'm very excited about where this will all go because it seems useful now and I'd be very surprised if this is the best that it ever reaches.
Absolutely, I totally agree. I edited my comment to add in more as I realised I'd not addressed that. It's surprisingly good right now, and given that it's useful that feeds into even more attention and money being spent on making it better.
Anyone not looking at chatgpt and thinking "what if this was even just a bit better? Or had better iterations or lookups?" is missing out on what I think is a revolutionary change coming probably sooner than people expect.
"Computer, please find out how GPS works and explain it to me like I'm five years old."
ChatGPT:
GPS stands for Global Positioning System. It is a way that we can figure out where we are in the world using special satellites.
Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so they can see everything on the ground.
When you have a GPS device, like a phone or a car, it can talk to these satellites and ask, "Where am I?" The satellites send back a message that says, "You are here!" and gives your device a special code.
The GPS device can use this code to figure out exactly where you are on a map. It's kind of like a magic trick! You can use GPS to find out where you are, or to find your way to a new place. It's super helpful when you're lost or want to go on an adventure.
The second half of that explanation is completely wrong though, isn’t it? There is no active signal being sent from your GPS antenna, no two-way communication.
The first half is wrong too. GPS has nothing to do with satellites being able to see lots of things from high up. The whole thing is just nonsense that looks plausibly like an explanation until you try to decipher it.
Achieving simplicity by glossing over details is distinct from achieving simplicity by stating something incorrect.
Which one is more appropriate and/or typical will probably depend on what questions the five-year-old is asking, but I think it's reasonable to say the former is usually preferable to the latter.
I didn't even notice that bit (maybe GPT is a flat-earther?).
Something GPT discourse has been demonstrating to me is that I'm not usually a very careful reader. I apparently skim a lot. Or maybe I skim GPT outputs because I'm biased in my expectations already?
Ian Calvert
Computer, please find out how GPS works and explain it to me like I'm five years old
Sure thing! GPS stands for Global Positioning System. It's a way to find out where you are on Earth.
GPS works by using satellites that are way up in space. There are lots of these satellites orbiting around Earth, and they are always sending out signals. Your GPS receiver is a special device that can pick up these signals from the satellites.
The GPS receiver knows how far away it is from each satellite because of the time it takes for the signal to reach it. By figuring out how far away it is from a few different satellites, the GPS receiver can figure out exactly where it is on Earth.
So, if you have a GPS receiver in your car, phone, or on a special device like a watch, it can use the signals from the satellites to tell you exactly where you are and how to get to where you want to go. Does that make sense?
---
I was also able to ask it to make it simpler and simpler and it did so pretty well.
And comparing these two results for the same question serves to illustrate how ChatGPT does not in any meaningful sense _understand_ what it is "talking" about. It basically just strings together relevant-seeming words in a plausible way. The result may be useful, or it may be nonsense.
You'd better know enough (and be alert enough) to tell the difference...
GPS doesn't talk to the satellites, but the satellites do send a message that allows the GPS to answer "where am I" for itself. It's not the worst way to explain things at that level, even if it's misleading.
About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
> About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
I think that's one of the biggest problems with using LLMs for accurate answers. A lot of the text that's very useful for modelling human language and questions is also full of factual errors and questionable opinions. Answers being wrong is a problem with old-fashioned web search too, but I think LLMs necessarily lean more on quantity rather than quality of content...
The benefit of web searches, and trouble with LLMs, is you can see the site that an answer is on and judge the credibility based on that. If I see that the answer is on Reddit ELI5, then I know that there's no assurance about how accurate the answer is. If it takes me to a page for a university engineering or math department, or the blog of a dude who built a receiver, then I can be a lot more confident that the answer is correct, though it might be technically over my head.
It is better at doing "translations" from one "language" to another "language". While it does know stuff and can answer questions about what it knows, that isn't its strength.
If you can do "here is the structured data, summarize it and make sure to present this information" it can do that quite well.
So far I've played with GPT doing a zero shot classification of HN titles and CSV hourly weather data to summary. In each case, it has done quite well.
Neither of those are things about what it "knows."
> this is probably appropriate for explaining things to 5 year olds.
I feel it's appropriate in terms of simplicity, but I think providing a simple wrong explanation is really only worth it when the more accurate explanation is substantially harder to simplify, and I don't think explaining how GPS works falls into that category.
Something like this I think is more accurate without being any more complex:
> Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so you can always see some of them.
> When you have a GPS device, like a phone or a car, it can look for these satellites, and if it sees enough of them it can use math to figure out where you are.
I’ll admit it was a decent answer. However, even a somewhat bright person (especially someone with radio / networking experience) could probably have realized that was an issue without any prior knowledge of GPS. They know that their phone has a GPS receiver, and that satellite phones accessing geostationary or high altitude satellites are much bulkier. Clearly, your cell phone isn’t transmitting to each satellite.
How will future LLMs be able to conduct that level of reasoning?
You can just as easily ELI5 this without spreading misinformation, though. Just change the third paragraph to:
> The satellites all broadcast a special code saying where each of them is. When you have a GPS device, like a phone or a car, it can listen to these codes and find out where and how far away different satellites are.
(Actually, this would be a fun thing to do with a class. Have a few kids march around the edges of the playground, saying things like “I’m next to the swingset now!”, and then ask one one kid to close their eyes, put them somewhere in the middle and ask them to guess where they’re standing.)
The explanation is misleading. The satellites don't see you and don't care about you. They are beacons sending a radio signal (the beacons says: i'm satellite X and here is a timestamp). The receiver based on the known position of the satelites and the beacon signals received figures out your position on earth using math!
So the explanation is either: satellites and their position or give the full explanation. Not this regurgitation.
Except for having nothing to do with how GPS works. The satellites essentially broadcast what time it is and your reciever uses the delay between broadcast and reception to calculate where/when it is.
Explain like Im five means break it down, not lie about the complicated bits.
Its terrible. This is totally not how GPS works, I.E. the terminal never sends a message to the satellite. This is the danger of early LLMs, a large amount of their output is false and there is no way to tell unless you are already an expert.
This makes them basically useless except for a toy.
Asking it to “explain how GPS works like I’m a seasoned engineer” yields a more detailed explanation and there’s no mention of GPS terminals sending messages back to the satellites.
Probably the “false output” you mentioned is due to the fact that the prompt asked to “explain it like I’m five”, which made ChatGPT answer with a “dialogue between terminal and satellite” explanation, which (arguably) may be better understood by a 5 years old.
I've seen this sort of response a lot lately, both for GPT and all the image generators.
Google, and all the other search engines, literally show snippets of the original content to display search results, and have had copyright infringement lawsuits over this.
Yet somehow, it's taken these AI, which generally aren't just block-quoting things unless you ask them to (and often getting it wrong if you do), for non-lawyers to start asking that question.
I think that just indicates it's more blatant. With search engine results I can click through to the original source. That's attribution, at least to the laymen. ChatGPT has no such thing.
I selected a few sentences from the given response, asked DuckDuckGo to find them, and got zero hits for any of them.
GPT definitely has some kind of internal model of things and how they relate to other things, and the capacity to express those relations in different ways.
Whether this is functionally the same as the model in my head, or some analog to it, or if such descriptions are merely cute but misleading anthropomorphisation, is far beyond my knowledge of neuroscience, psychology, and indeed machine learning.
Imagine that the GPS satellites orbiting the Earth are like the magic lanterns in the sky, sending signals down to a receiver (like the one in your phone or car). The receiver can use those signals to figure out how far away it is from each of the magic lanterns. By measuring the distance to multiple lanterns, the receiver can triangulate its position and figure out exactly where it is on the surface of the Earth. So even if you're a big, green ogre like Shrek, you can use GPS to find your way around and never get lost!
As I mentioned elsewhere, conversational AIs in the early 2020's are like PCs in the 1980's -- a new technology that still has lots of early limitations.
That is, if you can blindly trust the answers.. if it could provide the top sources for the information so I could fact-check it if needed, it would work
That's why in some respects, I think the sheer clunkiness of "according to website.com, claude shannon is..." or "ok, I've found this on the web..." in the current consumer tech is better than anonymous, authoritative somewhat novel answers than might be the internet's consensus opinion or might just be a vaguely relevant word vector structured like an answer
We're going to have a generation of people that are cock sure about things because ... ai, but they won't actually know jack shift. It sort of happened with google. People can figure something out but they don't systematically get things. Shallow explanation. Goldfish attention span
I think that assumes AI makes any difference to the self-certainly of humans.
My mother was into all kinds of new age and supernatural stuff. You may think you can see where this anecdote is leading, and oh boy I could go there, but instead prepare for a twist! One of the stories she had was a client saying "It is not possible, Susan, it is not possible that there are beings out there that are cleverer than what we is".
Is there a way to disuade users from blindly trusting responses? Such as unavoidable caveats, make no attempt to provide depth and instead strongly recommend reliable research?
And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
> And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
Don't think that's automatically true though: an AI can be consistently excellent at retrieving reliable sources and still get a lot wrong on its summaries. Also humans have their own opinions on sources even when they're not familiar with the detail, and finding out if the AI reads Nature or 4chan is part of the debugging process...
Sure. Really, I was thinking of the scenario where someone explictly asks for a reliable source i.e. "Thanks for summary about X. To whom should I turn for more accurate/detailed information?"
Oh yeah, it's spectacularly bad at some of that sort of thing. Ask for a list of academic papers and it'll make half of them up and give others new authors...
Agree. ChatGPT is how I imagined the internet would be back in the 90s (without having the foggiest clue how it might get there). Your questions are similar to the types of searches that people really want but have to translate to 'GPS simple explanation' or 'COVID in <town> this year' while hoping for a set of relevant pages in the Google return.
I've gotten it to fail on some word problems, but even then it was impressive to see it try to come up with a solution. More humorous were requests like 'Create a post for ycombinator about ChatGPT in the style of Chris Farley'. It's surprising and quite welcome that it even attempts that.
Can LLMs be updated quickly the way search engines are by crawlers and what not? ChatGPT is a couple years out of date and there was a lot of human involvement. Doesn’t sound like a quick or automated process. GPT-4 is still under development.
Au contraire, ChatGPT has already replaced search engines for me, at least for generalized knowledge searches. It provides better and more personalized answers to my questions than Google and is an excellent way to learn about new topics. It's not 100% accurate and can even be misleading, but that is hardly different than other search engines. Once more people catch on to how incredibly useful ChatGPT is its use will become ubiquitous (and we've barely even scratched the surface of what these tools can do)
ChatGPT is miles above Google search for me when it comes to learning the "hello world" of just about anything at this point.
From there it's a simple prompt of where to go next and asking for references to work that dig deeper.
Even if it's wrong, it almost doesn't matter because I can just say "are you sure?" given this extra information, and it corrects itself. Even then if it's still wrong you can prompt it in a ways to give you potential search terms to put into Google.
One of the challenges with ChatGPT compared to Google is that most employers, even the really paranoid ones, don’t mind you using a search engine to answer questions. That goes out the window for ChatGPT: My current employer won’t even let me use translate.google.com, let alone copy/paste some IP into someone else’s LLM.
I fully expect enterprise versions of ChatGPT to become widespread in the near-term. Modern tech firms already give their IP to dozens of third party tools and services, I don’t see why this should be any different
Also, in a further iteration, people could upvote and downvote results, or even parts of it, so that it learns what it knows and what it doesn't. If it just say "I don't know" for bad results or omit the bad parts, it would already improve things quite a bit. Doing so is quite simple, really.
Making a gpt query on every search I'd say that's indeed unlikely. Luckily, most queries are repeated, so caching can work very well.
There are many opportunities there, gpt could potentially be used for common queries to expand the results and even as a way to disambiguate queries. For instance, if I ask it:
"If I make a search query for "go", what are the possible different things I may be looking for?"
I get
> There are many possible things that someone might be looking for when they make a search query for "go". Some possible interpretations of the term "go" include:
>
> The board game "Go"
> The programming language "Go"
> The command "go" or "Go!" as a signal to start or proceed with something
> The verb "go", as in to move or travel from one place to another
> The website "GO", which is a popular search engine
>
> It's also possible that the person making the search query is simply looking for information about the word "go" itself, such as its definition, pronunciation, or usage in different contexts.
there may be better prompts, of course.
Also, they may identify some queries as being gpt friendly and get those through gpt, which they may also augment with a suitable prompt. The thing is, giving the query as is to the GPT model is not the only option. They can certainly be creative with how they ask gpt and interpret the results. They don't have to necessarily even display the gpt response, they can use it to improve the results.
Exactly. I keep reading all this hype and folks don't realize how much compute is spent on each ChatGPT query (I'm sure it'll be optimized over time). There's no free lunch
No, it isn't. It generates 1 token at a time in a loop until the response is finished. It's a highly serialized task. Parallelization increases the throughput of how many queries can be processed simultaneously, but you wouldn't be able to speed up a single query.
I really wonder how the economics of running chatgpt for Bing's request will work. GPT is anything but cheap to run, IIRC, it requires more than four A100 GPU cards.
Of course- it makes sense to run ChatGPT free now (reminded that it's an experiment) for training & betatesting. But running it even larger, and on a search engine ? It's a magnitude of difference imo, and totally different than a short-time bill for an experiment.
Who gives a shit how much it costs. The value is massive and 10x better than Google results. Are you comprehending what that means? After competing with Google for over 10 years with nothing to show for it, Microsoft is drooling over this technology.
There’s a very old story about Microsoft and Win 3.x pen computing. One of the leads was explaining that they worked on it, and someone else opined that it was too bad that they didn’t provide anything useful for MS. The lead replied that the goal wasn’t actually pen computing, it was a special teams “block that kick” play that kneecapped a competitor and so achieved its goal.
If MS can reduce the absolute firehose of money that ads provide to Google, it will totally be worth it even if it takes lighting money on fire to do it.
The curls have returned two distinct IPs. Both are google's. How probable is that for a plausible guess? Run the experiment yourself. There was an article a while back that shows the prompt preamble to get you going. See what you get.
["Overall, the specific details of the dataset used to train GPT-3 are not publicly disclosed, so it is not possible to say for certain whether interactive sessions by trainers were used in its creation." - the horse's mouth]
As I just said elsewhere it's not going to "run curl commands" in training time. it's trained on a corpus of public data, it's not running commands at all
Noticed this as well. Just like you can ask it for showing `uname -a` and others. The results tend to vary given the temperature of the result and just seem to invent realistic bits of data from the one it noticed in training (e.g., Stack Overflow ?).
The training technique doesn't require internet access as well, it's not going to give you the IP it used during training.
True, but it does not have to. ChatGPT is it's own thing. It has functions search engines can never accomplish. You can say the same thing the other way around, "Why Search Engines won't replace ChatGPT?" The essence of what ChatGPT is is here to stay. We'll be seeing different flavors and it will be a big part of everyone's life.
Googler opinions are my own. (I know nothing about search).
Google has LaMDA, which I could see them putting the answers from it into the knowledge box at the top of searches. There is no reason they have to be mutually exclusive, a search engine can provide answers from both. Google search is already an aggregation of multiple sources (images, web index, knowledge, shopping, video, flights, etc...). Adding another source seems like the obvious path forward (assuming accuracy and cost make sense).
Very true, they already have different sections in the results pages. There's no reason why they can't have one more. They can also use the box as a way to improve accuracy by having user vote on the quality of the result.
If we don't need to click a link does google receives less revenue? I think including more direct answers from a search query would require changes to Google's business model.
We really need you to stop posting so aggressively. If you can't or won't, we're going to have to ban you. You've been breaking the site guidelines frequently. That's not ok, and I warned you about this once recently: https://news.ycombinator.com/item?id=33860364.
I don't want to ban you because you've also posted good things and if you'd simply follow the site guidelines you'd be a good contributor, but the current pattern is a problem. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and fixing this, we'd be grateful.
1. ChatGPT gets tripped up in some edge conditions.
2. Search engines provide some interesting features that ChatGPT doesn't.
These are both bad assumptions looking forward:
1. It's evident that ChatGPT or other chat-based text generation will only improve over time. These kinds of quirky edge cases will _always_ exist, but will be harder and harder to contrive.
2. These sorts of features, categorization, images, etc can be discovered via conversation. And there's no reason to think these features are exclusive to a search product. Code snippets embeds are already built into ChatGPT, it's not hard to imagine other useful embeds.
Finally, I'd ask the author to think of it this way:
As more text is generated, and those pages end up at the top of search results, the quality of the search results approaches that of a text generation model anyway. Search will soon be the unnecessary middleman.
> ChatGPT gets tripped up in some edge conditions.
No. ChatGPT has no idea of the truth value of anything it produces. That's fine for writing fiction, but it's not an "edge condition" when searching for actual information.
I am curious if search engine is the same? It has little idea of truth value of what it produces, other than prioritizing known and well-referenced resources.
It refers to real web pages, so it's not making up results the way that ChatGPT does. The value of those web pages is assessed using an algorithm like PageRank.
ChatGPT hallucinates facts fairly consistently if the prompt is not worded correctly. IMO the prime benefit of search engines over LLM is the ability to provide validated information and opinions that are directly cited to experts. This is unlikely achievable in a differential landscape, so LLM are useful but will probably be best used in combination with standard search engines.
I think the LLMs are better if you want knowledge synthesized, assuming it can be done correctly (it is very confidently wrong a lot of the time). But if you’re trying to find something specific, often you want the complete work published by a human as is.
Yes - you’re in an embedded space that’s the result of effectively compressing knowledge into something higher order. LLMs are not grounded, so to ask it to ground something would not be reasonable. It’s not a lookup aka search engine where references are maintained.
I used (still use) Wolfram Alpha for the longest time. Back in the day, I was amazed that it could calculate a bunch of things that were way out of range for Google, etc. - and I always envisioned future search engines to be sort of hybrids, you'll get linked to the correct websites, but you can also ask them for whatever you want.
I don't need ChatGPT or similar models to replace search engines, I just want to use them in parallel.
I'm excited to see what Google does with their LaMDA model, now that Bing will incorporate ChatGPT.
In the days of the Romans, tech writers carefully explained all the reasons Hannibal would never get elephants over the mountains…
More seriously, Google seems capable of learning from and acquiring similar technology. In that sense, I think their market position will remain. But ChatGPT is a threat to Google in the sense that it’s already drawing search volume away for some searches for some influential people.
ChatGPT is much worse at some things than Google - timely relevance and over-confidence come to mind. But it is astonishingly better at a surprising number of search queries. Given the rate of improvement and the ability for young developers to plug into the ecosystem and make unexpected new stuff, Google has plenty to be concerned about.
One thing the article misses is that users don't just use Search engines/Google/etc to just search. We use it to solve a problem we have. Find the nearest X, Cheapest Y, Answer to Z. A few of these can be done with the localized data that ChatGPT has. Ofcourse, scaling it to search engine levels is an insanely hard challenge. But it doesn't need to go all the way in to do that. Selectively targeting certain portions of what we use Search for can easily get you to a lot of daily queries that you google. Imagine if ChatGPT didn't need to crawl but you as a Service/Business had to integrate with it, that would could evolve into something different from Search engines and yet eat Search engines.
Sometimes I just want to know something, not search for something. Up until now I've used Google for both, but lately I've just asked ChatGPT on things I want to know about, as if it were a conversation partner. It still lacks knowledge and one must distrust the information, but this is the beginning of something very big, like a new age.
I enjoy the ability to be able to ask questions without having the other party feeling either attacked due to their lack of knowledge or my corrections, but just be able to focus on the topic, even if it starts getting weird like it telling me about "clockwork energy" as if it were a thing which existed, without being able to name me scientists which have dealt with the analysis of this topic (it started with my question "could time a form of energy?" after we were talking about the different energy forms which exist. It had the idea that in thermodynamics there exists the concept of "clockwork energy" which isn't mainstream, but wasn't able to offer me additional information about it). It's fun, 100%.
It's doing what the voice assistants were supposed to: solve problems by assisting you with more context in a conversational manner.
Text to speech queries solve the problem of my hands being broken. ChatGPT solves the problem of my brain being contextually broken and I need more context, not higher relevant search results at the bottom of the screen after the ads that I then slog through to wind up changing my query and searching again like some sadistic freemium ad driven Sisyphean task that flies in the face of innovation to help me reach a contextual nugget.
The problem with chatgpt is that I can't evaluate the source it used to learn the information to see if I think it's credible, and chatgpt can't give me any indication of whether it's credible or not, and its answer will be just as authoritative in tone if it's completely incorrect as if it's correct.
I asked it for a schematic to connect an i2c device to a microcontroller, it told me to directly connect the device to the microcontroller. I asked it if pull-up resistors were required, it said yes, and then drew what it called "pull-up resistors" (describing them correctly) incorrectly in series between the bus and the device.
I know that's wrong, but someone who doesn't really know what a pull-up resistor does or why they are there would be at least somewhat confused by that interchange, and would be misled into doing it wrong. Meanwhile, I can type in "i2c schematic" to google and get thousands of correct schematics back to look at.
They might not replace them, but it fulfills a role that Google could do previously.
Try to find out what type of a spring is in a spring clamp using google? "What type of a spring is in a spring clamp?" Or use whatever query you want. The top 10 results won't tell me the answer.
Ask ChatGPT. It'll tell you (spoiler: it's a tension spring or a torsion spring. It's not a coil spring.)
8 out of 10 Google results are trying to sell me spring clamps. Seriously Google, do you not see that your search engine is FUBAR?
A: Spring clamps are typically made with a type of spring called a helical spring or a coil spring. These springs are made of wire that is wound in a spiral shape and are used to store energy and apply a force when they are compressed or stretched. In a spring clamp, the helical spring is used to provide the clamping force that holds the clamp's jaws together. When the jaws of the clamp are opened, the spring is stretched, and when the jaws are closed, the spring is compressed, creating the clamping force. Spring clamps are commonly used in woodworking, metalworking, and other applications where a temporary hold is needed.
And even after reading through first post and your post I had to do a double take to figure out it was wrong.
This is exactly why I don't bother with copilot - I tend to skim when something looks right.
Same thing when I'm reviewing code - I don't go into details - I just do high level sanity checks, if I'm familiar with domain I check for problems I anticipate, look for potential improvements where my experience matters.
If I have to go in depth I might as well do the thing on my own.
Google says "tempered spring" in bold. Doesn't sound like the answer you're looking for, but it was very clear.
The majority of the search results are for places to buy a spring clamp. This is because the overwhelming majority of people who search for that term are looking to buy them, not to learn about what they are. I assume this is true for most specialized parts and equipment.
The majority of humans are more interested in accomplishing a task than they are in any form of learning. So that's what Google optimizes for. They have billions of users and most of them aren't looking to learn about how anything works on a given day. They just need to buy a clamp, or whatever.
Well, I then did a google image search (which is much better than it's traditional search) and it came up with things that did look like that one I had in my hand.
But I do get where you're coming from, ChatGPT is way too confident when it's wrong!
Curious, the second result for me was aboutmechanics.com, which said it was a coil clamp. But I agree Google seems to be interpreting the question as "what's a spring clamp".
My understanding is spring clamps typically use a helical spring, which is a compression spring coiled into a cylindrical shape. Other types of springs, such as leaf springs and torsion springs, may also be used in some spring clamps, depending on the specific design and intended use of the clamp.
In your first post you also equated tension and torsion springs, but they’re very different from each other. The former stores and releases linear energy while the latter stores and releases rotational energy.
Either way, I don't know that I see it an example of ChatGPT "replacing" a search-engine: search engines never really claimed to be able to provide precise answers to detailed technical queries, they just do the best they can to find webpages that might contain information relating to your query. But I suspect it won't be long at all before Google/Bing et al will be able to give you answers to such questions at least as good as ChatGPT where up-to-date/live info isn't involved. At some point they'll stop calling themselves "Search engines" and just be a single entry point for any query you might have, regardless of where the information comes from.
but chat is not a search engine, so it will not replace them but it will make them obsolete. I dont see any of this obstacles as substantial. conversational systems will take us back to the original, "bladerunner" concept of technology as something that gives us answers, not something that gives us the books to find the answers
The main hurdle for openAI is that it must find a way to link back to the web so that people have an incentive to keep creating content for their training sets. But it seems this could be fixed if the model provides direct links to e.g. buy stuff in its answers.
> not something that gives us the books to find the answers
And what if you want more than the generic summary answer? You could keep asking questions, but at some point, you might not know what to ask or what the alternative answers might be, particularly on a controversial topic, or one where the answer isn't fully known. And there could be lots of reasons you might want to visit a site over just getting an answer. If search is replaced, something else needs to link to those sites. Maybe I'd rather watch a video, look at videos, or find a discussion.
Good point and that's why I recently came to an interpretation that search engines return some kind of "tables" for us (but not in the tabular form). List of links are rows, and for every link we have properties ("columns"): the title of the page, the date, the snippet with phrases containing our terms. The same is true for stackexchange.com/stackoverflow.com pages, the answers are rows, the text, score, comments are properties of this row. We are used to scanning this "tables" either for finding best answers or exploring multiple variants of answers. But single answer for everything is too limiting.
Yeah, I agree. I do think Microsoft is capable of messing it up and blowing their first mover advantage. They have messed a whole lot of things up over the years. But they haven't messed up Github, VS Code, Typescript, several other things (IMO anyway).
It'll be interesting to see how hard they try to stick to the Bing vision versus what they should do, which is replace it with something conversational.
With maybe one exception, search engines are already replacing themselves. They're "answer" engines now, but that's really a pretense for stealing content and telling the user what their interests are. While that might line someone's pockets, turns out that's not really useful for much other than the casual normie trivia question.
ChatGPT is an actual question engine. Even if ChatGPT doesn't live on in its current form, the institution of ChatGPT is the future of asking the internet a question and getting an answer. That and many other things, including (non)human relationships.
I hope it succeeds in making The Google's questionable value proposition all that much more clear. Yet if it goes down the road that The Google went, things can become much worse. I can't imagine governments are going to just sit back and relax while the next generation of manufacturing consent is manifesting before them.
Relatively soon someone will figure out how to automate having chatgpt stand up barely functional (over time) servers. Then they will be able to ask chatgpt to create websites about x, y, or z. At that point, knowledge and our ability to trust the internet, which was already on shaky ground, will completely evaporate.
I wrote in the input: What is the name of that Nintendo game with animals and they live their life and do stuff?
ChatGPT: It sounds like you're thinking of the game "Animal Crossing." In Animal Crossing, you play as a character who lives in a village inhabited by anthropomorphic animals, and you can do a variety of activities such as fishing, bug catching, and decorating your home. Is this the game you were thinking of?
Google: Animal Crossing is a social simulation game, dubbed a "communication game" by Nintendo. It is open-ended, and the player's character can live a separate life with very little set plot or mandatory tasks. Animal Crossing (video game) - Wikipedia
ChatGPT-like question analysis is already integrated into Google search engine, for the good and for the bad.
Google is just a lightning fast but messed up index combined with really good graphs.
ChatGPT now is only one tool
Soon it will be just one piece in a box filled with a combination of tools that let you just know whatever you want to in a depth of detail you specify it to be laid out:
- large conversational language model that generates the piece of information
- fact checking model that applies post checks on logic and consistency
- recitational graph model that backs the propositions with sources and ratings
- geospatial and locational model that adds references in space
- recognitions development model that can classify how realizations and knowldege develop over time
- social trend model that can adjust to conversations of the society as a whole
All this in combination will bring any knowledge in any level of detail with a defined level of certainty.
Here is a good recent overview of where automated fact checking is positioned right now: https://aclanthology.org/2022.tacl-1.11.pdf
Fact checking of AI generated content can potentially be optimized for this use case.
And for speed: I think many of the computations for several layers of AI processing can be decentralized in the future and re-establish the heart of the Internet. The contributions for such computations need a high level of trust and therefore a public immutable ledger might be a viable approach. So perhaps decentralized AI might be the use case where finally some "blockchain-crypto-cringe" might actually be useful ;-)
We need to be aware that we just entered the steep part of the exponential growth graph.
Amazing times ahead.
I'm blown away at how many tech savvy people on hackernews still persist with Google search or talk about it like it's the only choice when there better and/or equal less evil alternatives.
I say this as I would expect more use of alternatives here of all places.
I've been using ChatGPT to suggest me books similar to books I've liked and so far the results have been good. Many of the books that are recommended are books I've read and enjoyed so the remaining ones on the list are added to my reading list.
My biggest worry is that search engines combine ChatGPT like functionality with the existing ads based business model. Combing ChatGPTs persuasive/confident responses with brand bias feels socially destructive.
I've found that if I give it a pair of questions ("Why does X breed faster than Y", "Why does Y breed faster than X") it'll give me two neat little responses that explain why both these statements are true.
That doesn't sound useful on the surface of things, but the additional information in the responses allowed me to craft more traditional search queries I wouldn't have thought of (in my example I learned that species X is hermaphroditic, and that water temperature may be having an impact).
So replace? No. Enhance, if used carefully? Maybe, for now.
These feel like really strange criticisms, since they're just so straightforward to resolve.
Can't figure out typos? That's a solved problem. Doesn't understand all questions correctly? It's going to get much better at that very, very quickly. Chat results don't give context? You can put AI search results in whatever UI you want.
Not shocking to hear a search company say AI won't disrupt search, but if this is the best they can do, I feel pretty confident saying that AI will disrupt search very soon.
When ChatGPT first arrived, it felt amazing and revolutionary. Since then it's been so neutered that while still feeling pretty cool, it's settling into gimmick level :( .
It won’t have to replace; it can just augment and be very successful at it. I’m just unsure how to cover the massive costs of running this thing with millions of req/sec.
In my opinion, ChatGPT is a big threat for close-ended questions: What is a car?
What I am not so sure about, is this really the majority of searches? I assume a big part of searching is about discovery — finding articles, products, images, etc.
For that, you need a good ranking and efficient crawling. Two things ChatGPT doesn't have.
Given the strong brand of Google, only time will tell if ChatGPT will eat a large chunk of Google's market share.
You're seeing shorts on YT is because Google has changed their ranking algorithm.
It seems that the quality of shorts has really declined lately. It's almost as if they are being made with the assumption that the only audience they will have is comprised of members of the gen-z generation, who are known for having short attention spans and being easily captivated by quick, sensational videos. These types of videos often feature someone doing something seemingly ordinary, like crossing the street, but with a dramatic caption promising that the viewer won't be able to believe what happens next if they continue watching. It's truly sad to see the decline in quality of these shorts, and the reliance on cheap tricks to grab and hold onto the viewer's attention.
I'm using chatGPT for some
repetitive things but I have to double check the answer every time just to be sure. There's also the reality that you need to literally know what do you want and tell that to chatGPT, then analyze the answer and try to incorporate it in your flow and, in some environments, double check with legal just in case.
ChatGPT already replaced search engines for some (about 3-8%) of my questions.
Search engines really suck at answering complicated questions that use the same word in >1 context. For example ChatGPT answers this with ease (whereas google doesn't get close):
> Are there any currencies which have two or more units of currency and whose major unit is comprised of more than one hundred subunits?
Google already provides a chatgpt-like experience with the questions it displays based on the query. It is unlike chatgpt in that google comes up with those questions but they are usually spot on and you can see the source where the answers are coming from. Perhaps all google has to do is provide a chatgpt like interface to that subsection of search results so it feels more interactive like chatgpt
I am absolutely sure that if Chatgpt created a google like interface on a domain like askme.com. without a required login, it would become immensely popular. Possibly eating away a lot from traditional search engines.
Google search is becoming the new Yahoo. Its becoming bloated if not already. And its just time for a new era in which a chatgpt like product perfectly fits.
This feels like what a company that hasn't accepted the future would say. Yes, ChatGPT doesn't do exactly what Google does. Can it be augmented by well understood search retrieval engines to generate a much better response? I think for a lot of cases that seems to be undoubtedly true.
I'm (the author) actually in agreement with you. LLMs are going to be a big part of search in the future. I alluded to that I'm the post. I'm less convinced about search as a chat interface. But LLMs for query understanding, ranking, etc.? Of course.
How do you see LLMs handling the constant flow of new data, given spam, SEO hacking, fake news and what not? Can the training be reliably automated and performed that quickly? It seems the chatGPT folks are allowing free access so they can test and fix abuses with human intervention. But that's hard to do in real time.
ChatGTP maybe, but OpenAI hasn't even tried to train a model to replace search.
Until somebody tries to fine-tune a model using RLHF explicitly with the goal of replacing Google it's very hard to know what the resulting experience would look like. It could be shocking if ChatGTP is any guide.
I’m a bit surprised that a product manager for a search team would put forward arguments like this. ChatGPT isn’t a search engine, but that doesn’t mean it won’t be a significant part of a search engine product.
Query correction is a common component in search engines. “Did you mean” is everywhere. It is just one component in a search pipeline.
The stupid simple integration I’m sure Google will release is to put ChatGPT on top of the search results rather than the query itself. The question at that point is the economics of it: how much does it cost per query, and how much extra value can Google derive (or avoid lost value i.e. lost customers).
This is an advantage that Google has over upstarts like Kagi and DDG: Google can keep their cost per query much lower since they don’t buy their results from other sources.
Algolia's main product is search related. You can tell from this article that they are scared about ChatGPT even partially replacing search. They are very scared of losing business, and anyone who is scared of ChatGPT will be writing articles like this, badmouthing ChatGPT.
I remember when Google first arrived on the scene -- I had been using Alta Vista before then and Google's search was like a miracle. But Google these days just feels like Alta Vista in the 1999, except that the haystack is a lot bigger now and more of it is spam.
But will it replace search engine “frontends”? You -> ChatGPT -> Search Engine. Any examples of Chat GPT querying data live? I’m no AI expert.. but it feels like this would make a lot of sense.. it’s seems like what current voice assistants are doing right?
Agreed that chat-based search won't replace traditional search any time soon. BUT even in this article, the author mentions that hybrid semantic search is actively being integrated into Algolia.
Both have pros & cons and the best solutions over the coming years will be hybrids.
"That will NEVER replace search engines!!" -Search engine vendor
"Those things will NEVER replace the horse and buggy!" -Buggy whip vendor
but seriously. Search engines are (1) a virtual monopoly for one company. (2) hopelessly conflicted due to being run ONLY in an ad-supported model, just because that early business model was never revisited. (3) prone to a neverending trench warfare against spam and black hat SEO.
Not that AI based replacements for search might not end up with the same flaws, but for the above reasons, I'm rooting for whatever comes out of OpenAI to completely upend Google's current business model (whether or not GOOG ends up being smart enough to be one of the leaders in that space or not).
Just heard of this, so I logged in with my google account. The mistake it makes below is interesting. If this were a human response, I would assume a brain fart. Not expecting this type of error from "A.I."
Me> Find the area of an ellipse with semimajor axis 5 and semiminor axis 4.
ChatGPT> The area of an ellipse with semimajor axis 5 and semiminor axis 4 is 40π. You can find the area of an ellipse using the formula: A = πab, where A is the area of the ellipse, a is the length of the semimajor axis, and b is the length of the semiminor axis. In this case, the semimajor axis has length 5 and the semiminor axis has length 4, so plugging these values into the formula gives us: A = π(5)(4) = 40π.
The trick is to get it to show its work first and then calculate the answer. If it guesses the wrong answer first and then writes out the steps, it won't correct it.
This works: "Find the area of an ellipse with semimajor axis 5 and semiminor axis 4. Don't calculate the result until the end."
After computing your example, I tried my original question which now gives the correct answer. In fact, it gives essentially the same response for both forms of the question. I guess the training works as it should.
Being not very good with numbers is one of the limitations of the current model. I imagine that in the future we will see integrations with a number of different tools to overcome these limits - WolframAlpha being the first one that comes to mind when talking about algebra and calculus.
Honestly, for many simple programming tasks/questions, I find it’s already rivaling Stack Overflow and Google for quick results. Just last night it flawlessly wrote me a scaffolding for a Bash script that parses specific command line flags
I have experience with large NLP models and here are my humble thoughts:
In a large language model you can refine your original request via conversation. In a search engine you tend to rewrite your query to refine it.
LLM conversation is much more stateful than an almost stateless query to a search engine.
However a search engine shows you more than one answer.
With LLMs you can't easily add knowledge graph style facts (subject predicate object). I has either memorized a fact from the training set or not and you can't easily insert one more fact in there. retaining LLMs is not cheap by the way.
An ideal search engine for me is conversational, up to date and one that gives you multiple answers.
Article claims that chatgpt is overly sensitive to queries eg -gate suffix and come lad vs cum laude. This seems like a weak argument based on a temporary shortcoming. Any NLP methods search engines use for word meanings should be trivially integrated into chatgpt.
Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I'm sure the author is better informed, but what about the scalability of search? I'm not sure if the neural network approaches are as cost efficient (today) as some of our current search algorithms when it comes to answering queries.
Author here. I was surprised to see my post on here while scrolling!
My point about those examples is less that ChatGPT got it wrong, but that it's impossible to know really when it does, because it seems so confident and you only get the one result. To be fair, this is a problem with webpages, but you at least get multiple choices.
> Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I can see why you'd say that, but that wasn't the goal. I originally wrote this as my own POV for my own blog. I really do think there's a UX problem in this (and, as I mentioned, I am a big fan of recent LLMs).
> [What] about the scalability of search?
You're right, the NN aren't as performant as lexical search. They're getting a lot better though. (We're actually working on this at the moment.) LLMs, though, have a ways to go, so it's hard to use them right now for real time search.
I'm sure ChatGPT has it's uses for some people, but the few times I've tried to use it for tasks I would have used search for it's been confidently, eloquently wrong. A search engine you can't trust, or at least evaluate it's sources, is completely useless. Recent publicity on LLM has been incredibly successful at over-inflating the hype, largely because the technology fulfills the fantasy of being able to interact with technology in plain language and get plausible, seemingly coherent responses.
When they actually commercialize ChatGPT as a search engine, and when enough people get burned on nice looking answers, it’s gonna get relegated back to hobby/fun thing to use.
There are two things I have been wondering about ChatGPT:
1. What impact will it have on search engines? With ChatGPT, it will be easy to automate the creation of websites, which will be referenced by search engines, right? So maybe someday half of the results will just be generated websites that may or may not say complete nonsense in an authoritative way.
2. Will the successors of ChatGPT learn from websites generated by ChatGPT, too? What happens when the new model learns from outputs from previous iterations?
ChatGPT will make search engines useless. But not by replacing them. It will produce so much and so high quality SEO spam websites, that you won‘t find any real content anymore. It will increase the noise level in the internet so much, that automatically finding a signal with search engines will not work anymore. We will be back to manually curated lists of webpages as in the good old days of Yahoo.
It seems like an article from Kodak or Nokia. It's only a matter of time, and it will be sooner rather than later, that multimodal LLMs will replace search engines. All of the problems mentioned in the article are being addressed as we speak.
What's going to really slow their adoption down is creators of the raw information on which they are trained are going to need some way of being paid for their work.
Until payment is figured out it seems likely that creators will increasingly restrict access to their information and that will impact the utility and adoption of the models.
Google's probably got the advantage in terms of figuring that side of it out if it can overcome its inertia.
I give ChatGPT a question on how to do X in some programming environment. So far same as googling. But then ok, for me that solution gives errors, or function doesn't exist, 2nd question to ChatGPT, bam half the time it gives something that puts me on the right track. I can't tell google that my python/postgres problem is special, so please do That example, but with such and such restrictions.
why the hell would anyone trade a bunch of factual sources, documentation, and actual expert opinion for an LLM that's consistently and confidently wrong for a significant portion of queries?
I can see this thing being useful as a summarizer. Even then I don't trust it to be correct and have to fact check every single thing that comes out of it against results I find in a search engine
You know what's even better than an LLM? Actual experts (bullshitters excluded). Having books written by people leading in their respective fields, professors, managers, people above you who have experience and know what the hell they're talking about. Why? Because they understand what you need to know in that context
I'm probably being too hard on chatgpt, I'm sure it will find a commercial use that's beneficial to society. I just hope search engine providers don't hop on this fad and raise a generation of people educated on word salad masquerading as authoritative information
For any chatbot I will ask only 2 questions, one is about myself (since I placed an easter egg in software AOL took and used back in the day) and what time is it? ChatGPT thankfully may or may not be connected to the internet @ prompt but that is usually a good giveaway. IMO Google blew it when they took away Google via SMS, this is the hole ChatGPT(x) can fill.
Thanks for sharing that. I am wondering are there situatios today where you would have SMS but not have web access? Or is there somethng else appealing about SMS search?
Have they not heard of it? The web is dead and as the poet said, giants will roam the earth once more.
Every time I interact with davinci-003 it makes me feel more and more like the first real paradigm change is coming. We are still using our computers and writing our programs in the same way as 70 years ago, I emplore you to read 'structured programming' debates and see how we argue about the same things now. Every business product basically still competes with a word processor and spreadsheet. We have made hundreds of thousands of lines of code so we can auto focus the next field on a form, and it still doesnt work on 99% of the cases. The GUI was not a paradigm change, it was more of the same, maybe even worse, mobile overpromised, anything besides video is just worse desktop.
Lists and tables.. for 70 years
This is the first real change.
--
"And what doeth the saint in the forest?" asked Zarathustra.
The saint answered: "I make hymns and sing them; and in making hymns I laugh and weep and mumble: thus do I praise God.
With singing, weeping, laughing, and mumbling do I praise the God who is my God. But what dost thou bring us as a gift?"
When Zarathustra had heard these words, he bowed to the saint and said: "What should I have to give thee! Let me rather hurry hence lest I take aught away from thee!"--And thus they parted from one another, the old man and Zarathustra, laughing like schoolboys.
When Zarathustra was alone, however, he said to his heart: "Could it be possible! This old saint in the forest hath not yet heard of it, that god is dead!"
Maybe the first version but I can see several iterations later where this technology makes Siri and Alexa actually Wizard-of-Oz levels of smart where knowledge based tasks that used to require search engines and multiple queries to refine your results will no longer require this level of user input.
With an approachable interface I can see it replacing a large chunk of search extremely quickly. Lets be honest here: humans cannot even settle on what the "truth" is so whether or not truthful answers are returned is probably less important to the success of such a service...
And yet this thread is full of examples of people using ChatGPT instead of a search engine.
Phrasing the problem as a binary is wrong. The correct framing is “what is ChatGPT good at today? What are LLMs likely to be good at over the next 5 years?”. It isn’t everything. But it’s a hell of a lot more than nothing.
I want to add here that I'm building an app at the moment with code solely generated through prompts for ChatGPT. So far, except for a few issues and manual refactoring, it has done very well.
ChatGPT even apologized when it suggested proprietary Firebase, when I told it to use Supabase instead.
Isn’t the main benefit of chatGPT that it combines multiple resources to weave the answer? Things like handling typos or formulation nuances sound auxiliary to me, probably non-goals at this stage. It’s still less than 1 month old..
Why does every game have to be zero-sum? Why can't chatgpt live alongside search engines as just another category for results? Like "Images" and "Maps"?
In fact, I look forward to Bing integrating with chatgpt.
Every single time I've seen this claim made and the person shared their methodology for searching on Google, it becomes very apparent that the problem is not Google, but the user.
That’s a bit harsh. Early Google didn’t really require any special skills or knowledge on the user’s part. Current day Google requires hacks and workarounds to avoid both SEO spam and Corporate Google’s attempts to sell you something.
How long would it take for "ChatGPT farms" to crop up and try to influence the results? How much harder will it be to "clean" the results from any learned data set?
is soon a year or a decade? the author says he's at his workplace for 7 years, I think chatgpt 2030 will absolutely destroy google search 2023. the question is what google search 2030 will be?
I can see that being true for the top 5% (arbitrarily chosen small %) of difficult translations with professional requirements, but I think for the vast majority of day-to-day translation between the major languages (maybe only within the same language family), it is already sufficient and improving.
I only have the anecdata of my friends and myself, but we have yet to encounter a casual conversation type translation that DeepL or Google translate have failed on for Spanish <-> English <-> French. Granted, it doesn't always return the most optimal translation, but it has always given the intended core idea.
ChatGPT might not replace search (entirely), but for many many use cases, it has already replaced search.
When pure machine translation was starting out, there were lots of similar posts about it how it can't YYY, but in short amount of time it has become very effective.
ChatGPT for me is a much better StackOverflow. In fact, I'd rather people be asking and answering SO style question in a format that is easier for machines to understand.
ChatGPT is better at searching for a concept, if it was unencumbered, I could better constrain the concept space it was searching over.
Precisely my experience with 95% of stuff I google.
I don’t know if that’s just the nature of the things I usually search for (primarily technical subjects where the query shouldn’t be “dumbed down” and reinterpreted or you get a completely different meaning), but I’ve become incredibly dissatisfied with Google over the past few years. I imagine it’s a fine search engine if you’re looking up the URL for Netflix.
I'm curious what will happen in future systems where the AI can continually learn and update it's model on the fly. I note that ChatGPT can't download a URL from the Internet and summarise it for example, or apply it as new knowledge. Of course it seems ripe for exploitation since there is so much misinformation on the Internet anyway.
Outdated? How so? The primary thing Algolia is guilty of is being massively overpriced to the point it’s a non-starter for any platform with even moderate usage. But their full-text search functions quite well and is very fast.
I think this was framed around using google to answer a question vs using Chatgpt.
For example. Asking "How can I tie my shoes?" to google requires that I click on a link (or multiple) and hope for the best. Meanwhile Chatgpt is likely to give me a detailed walkthrough and answer any follow up questions.
Another good example someone mentioned was "How did ww2 start?". Google gives you a path to find the answer, but Chatgpt will outright give you an answer.
I'd say Chatgpt's competition is really google assistant and siri vs google search. Personally, if I'm using "Okay Google", I don't want to be referred to links as that requires me having to pickup my phone.
I don’t know about the author but for me personally chatGPT has already replaced 70% of what I used to go to google for. More importantly it saves me lot of time. That is just incredible.
ChatGPT isn't search! For gods' sakes! It's not even related to search, ChatGPT threatening search is a meme put around by VC tech bros. Which is ironic - because ChatGPT is a tech bro - it knows about everything, talks authoritatively, and is mostly wrong.
The only thing to worry about in this entire conversation is how search is broken. Search is so broken, that we're willing to just replace it with some tech bro. That's what we have with ChatGPT. Oh don't Google it, just assume some tech bro instead. What's the likelihood the techbro actually knows the answer? Pretty low.... but he'll sound very convincing.
Search is broken, and if ChatGPT is the answer, I dispair.
At the same time, ChatGPT has frequently impressed me, not with everything (my expectations are reasonably low) but it has performed amazing work for me (typing out form letters, code language conversions).
For what it's worth I wouldn't use ChatGPT for search like I do with Google, but what it has done is taken away time I would be Googling for things like "how to write X form letter". I expect as it matures, it will take more time away from me Googling.
All these takes underestimate the following:
1) How quickly ChatGPT and its ilk will advance to solve relatively low hanging fruit like "ChatGPT is wrong about this one thing". The delta is extremely important here.
2) How slowly the Google bureaucracy will grind when releasing anything remotely like ChatGPT. All the committees and the burdensome processes in place in Google will keep this new technology locked up for years, and ensure that the final result is a camel (horse designed by committee). It doesn't matter if they have superior technology if they never use it or release it.
3) How much Search means to Google will mean they will treat any product changes to it extremely carefully while Microsoft will be willing to experiment with Bing like they have with Co-Pilot and GitHub.
Personally, I wouldn't go long on search engines that don't have a strong ML component to them in the future.