Interesting to see the amount of negative comments.
Most of the negativity seems to come from the following points:
- EU-funded project cannot succeed in tech because previous EU-funded projects have failed in tech in the past (and generally government-funded project in tech are suspicious)
- Search is hard, and therefore it will fail
- The project is underfunded
Even if the points above can all be valid (though the obvious US-funded startup launched by a bunch of uni students seemed to have fared pretty well) it seems we are missing the point of this project.
The proposal is to contribute to the creation of open building blocks necessary to enable others (including private US companies) to make better search products.
Better search product are needed.
Shall we remember HN of some of the intense conversations that happened here about Google failing us:
- Google Search is Dying [1]
- Every Google result now looks like an ad [2]
- Google no longer producing high quality search results in significant categories [3]
So while, yes search is a hard topic, we should welcome initiatives aiming at improving the ground infrastructure needed to lower the barrier to entry on this subject and hope it will allow many companies to build better search products (or inspire other initiatives to contribute in similar and even more successful way)
One reason search is hard is people are very motivated to game your system. That makes being transparent about how it works a fool's errand. It also isn't clear how the economic structure works: improving search relevance by x% is tremendously socially valuable, but probably makes Google's bottom line go up by a thousandth of x, and they have a very direct understanding of the connection. Without that money, how will this get the resources to succeed given the adversaries are learning from the people with a lot more?
This sentiment is common, but is largely based on Google (and similar commercial actors). I'm not convinced it generalizes as much as it seems. It is uniquely profitable to manipulate Google because Google's outlook of the world informs not only its search engine, but its advertisement business.
> improving search relevance by x% is tremendously socially valuable, but probably makes Google's bottom line go up by a thousandth of x, and they have a very direct understanding of the connection.
I'm not sure this is correct. In a vacuum of real competition, the most profitable ought to be how it is right now, when search results are kind of ambiguously bad, so you need to click and skim through a few results to maybe find what you want, multiplying the number of ad impressions.
It's nothing to do with Google advertising. It was a problem before Google had advertising. Being the number 1 non-ad result for a search will get you many more hits than being a number 2. If you don't appear on the first page, you may as well not exist.
Google's priority with ads is to make sure ad buyers are happy with the results they are getting. This means making sure the people who click the ads go on to buy the product being advertised. That's what ad buyers measure.
Being result number 1 not result number 5 matters a lot to companies that are selling things to people who are searching. The value of the ads comes from that property, not the other way around.
Indeed, and that's why search relevance is important.
We can talk about the Importance Of This until we die of old age. Google failed us, Google is Borg, Yandex is FSB, Bing is ... Bing, etc. However, the fact that there is a problem to be solved and It Is Important doesn't mean that the EU will solve it. If anything, it's just the set-up for another political play that will have damaging consequences to the internet as a whole, just like GDPR, EU's poster-child "internet project".
They made GDPR just strong enough legislatively to be annoying, but not strong enough to actually change anything. Companies can still store EU citizens' data anywhere they want and do whatever they want. There's no insight into this. It is an unenforceable law and the only artifact of it existing is that I have to have "I don't care about cookies" installed, so that Avast Antivirus can eventually decide to silently collect and sell on my data.
For all intents and purposes OpenWebSearch is most likely not meant to succeed at anything either, and is just going to be a political stepping stone towards legislature that will be awkward and make the internet worse for all users. EU has a long history of creating or hanging onto laws that betray a misunderstanding of how the digital age works, how the internet works, how data can be copied or moved around for free.
Here's an example. Every country in Europe has some sort of legal construct in place that will prevent you from secretly recording a conversation you're having and uploading it to the internet. So for example, take Austria. They accomplish it by preventing you from publishing it on the internet. However, they don't prevent you from recording it. There are laws against secretly recording a conversation you are not a part of, but there's no such law for the situation when you are part of the conversation. So you can record and upload, just not publish. However, you can get on a train and go to a different country which has laws that prevent you from recording, but has no laws against publishing. Then you're in the clear. Or you can just use a VPN so that it looks like you uploaded and published it from the other country. Or you can just upload to YouTube, which will not show where the thing was published from - and claim that you did so during your tourist visit to Vietnam or whatever. And if someone brings a civil lawsuit? Good luck trawling the Vietnamese legal system for clues about that specific issue. Hope you know this rare language with virtually no legal experts who at the same time speak Vietnamese and your particular local European language. The costs would be on the order of tens of thousands at least, which is out of reach for anyone but the wealthiest EU citizens.
Want more? Egregious copyright related laws are known by everyone.
So are dns blocks of shunned sites. Or the recent Austrian project to block cloudflare IPs which literally broke the internet.
The fact such laws are still in place from before the internet - or are even still being put out - and are effectively unenforceable while making the internet worse for everyone - makes it clear that the governments in place simply have a misunderstanding of how the internet works.
None of that will stop politicians from coming up with BS excuses breaking the world with fever-dream laws in order to push their latest political agenda. The real question is: if such clearly unfit legislature is being put in place for political folly in industries we understand - how much of that is happening in industries we don't understand? Health Care, Food, Agriculture, Civil Engineering, Patent Law, and so on. My guess? You can tell what my guess is.
As for Open Web Search, what everyone should really be asking themselves is: ok, so what's the scam that's going to be pulled here?
> As for Open Web Search, what everyone should really be asking themselves is: ok, so what's the scam that's going to be pulled here?
As I'm one of the people who is working within the Open Web Search project please allow me to feel strongly about your statement. I've been involved in campaigning for this project since around 2014. This project did not originate as a 4d-chess move of some political game. It exists because of the hard work of a group of people, some of which are researchers, some of which are working at smaller search engines and some of which are involved in civil rights organisations.
Currently still sitting in the kick-off meeting I can tell you that we are actively discussing how to get this project to produce useful results for a european open web index. Getting the EU to draft problematic legislation is neither in our power nor in our interest.
"One reason democracy is hard is that people are very motivated to game your system. That makes being transparent about how it works a fool's errand."
That's a major fallacy you're opening with. If something is important to us (as a society), we will find ways to make it serve us well even when it's under attack. The rest of your post even makes a similar point: the social value of well-working search is greater than the economic value for even the biggest search monopolist on the planet. So why not socialize it?
Well, on one side we have zero examples of open engines actually working well, on other we have long nuanced history of SEO vs search engine fighting unending battle, with SEO side trying to circumvent every way the search engine is trying to stop them from poisoning the search results. And barely edging out on that.
Being idealistic about it won't change the outcome.
Altavista was initially a demo of the capabilities of DEC Alpha CPUs. It was initially amazing. It got taken over by SEO rubbish, and beaten by Google who invented PageRank to get good results again.
PageRank was initially much less prone to SEO shenanigans because it relied on signals from other sites (incoming links) to decide how important a result was. Of course, as Google became more popular, people started sharing links on other pages and so on to cheat the PageRank algorithm. And Google have been caught in a fight with SEO ever since.
Democracy has some intentional opacity though. Consider the impact of making your personal voting choice public.
AFAIK, nobody has ever tried that because of the dangers of vote buying and coercion. Essentially, gaming the system.
(I don't think this proves anything! Simply wanted to suggest that comparing search to democracy doesn't significantly change the analysis wrt opacity.)
Exactly, I think the failure of such project is embedded in the incentives. OpenMaps are good but you’re not going to convince Google to make them default on Android.
The correct strategy isn’t to hand this over to a giant bureaucracy, but to create an atmosphere where we can have a dozen alternatives to Google.
This does seem to be the goal of the project. They mention that they want to make a centralized control panel and crawler, where you can control how your website is crawled and when
But at the end of the day the resulting dataset would be sold to third parties so they can rank the results appropriately. Which to me seems to be the only sane way forward. Only a government could run something the scale of the Google crawler and succeed at doing so. And then everyone can build search engines on top of that.
> And then everyone can build search engines on top of that.
That bit troubles me. If the index is maintained by a government agency, and every search engine is using the same index, then that's a massive censorship avenue. I wonder how "open" Open Web Index is going to be.
The only way you can beat google is to create a search engine in a small niche where they can’t compete, then eventually over the years expand into general search once they can’t catch up (innovators dilemma basically).
The only search product that comes close to this is Amazon search or booking.com (or maybe YouTube).
All multi billion dollar businesses. And when smaller ones emerge like the flight scanner one, google buys them.
I don’t see google going anywhere soon. They have a lot of faults but they’re still too good.
In Austria there is "geizhals.at" [1], which is a price comparison website for electronics, and they have the best parametric search engine I have ever seen. It's 1000x better than Amazon search.
However, there's no way Geizhals will ever expand into general search. The reason why they are so much better than the competition is that they are focussing on a small, profitable niche and presumably use manual data entry to ensure they have the best data.
This is HN and we're not even pretending to be optimistic about beating incumbents, but it is much deeper than that - there is also a profound sense of defeatism directed towards European entrepeurs as well as the world at large. It is an indictment of the current nihilistic zeitgeist.
Well, it is a forum run by a Silicon Valley VC firm … so of course there will be a bit of “only SV can do anything interesting in tech” gatekeeping going on.
Absolutely nothing, Amazon search is horrendous. Search by price is completely broken (and has been for years). Filter by exact phrase is non-existent. No way to search products by size, and on and on.
That's why I don't let my kids watch YouTube on their own... That said, as a European I'm more triggered by all the gore and violence rather than some naked skin (although I also wouldn't want sub-12 year olds encounter explicitly sexual stuff). The later seems to just lead to questions and adult conversations, but the normalised violence the kids seem to act out.
I second the violence thing. I has somehow become the new normal in the teenager world which is very disturbing to me. In my family we strictly use only Firefox with ublock for youtube (and the rest of the internet too). The difference is night an day. I always recommend it to friends of mine and every time I get positive feedback after this.
Once or twice I didn't use Firefox with ublock origin on youtube, and since I am pretty allergic to any ads as they insult my intelligence I just closed the whole page before getting to the content.
How can otherwise smart people accept being treated like brainless idiots by similar services is beyond me. And how can they teach their kids that's fine too is something I'll never understand... but hey its your kids, everybody sees quality of their parenting first hand (and then complain how kids are unruly and eat junk these days... gee I wonder where they took inspiration and who built their character).
I have YouTube premium. Everytime my kids get exposed to even age appropriate àds it generates attraction to crap. Even in magazines for kids.
So I just keep ads out of their system and they seem happier and more internally satisfied
I've recently been searching for some how-to videos and the number of ads is insane - but somehow they don't show when I watch in the video search results in DuckDuckGo
Same here, not using YouTube's built in search function eliminates most of the ad and also recommendations problems. I use You.com since they additionally show Reddit and TikTok results which I sometimes find more useful than YT videos.
One more reason to use an ad blocker for YouTube. I even use on my (android) "smart" TV. The open source app I use is smartyoutubetv. Occasionally it gets broken, but then a fix is usually an update away. It skips not just ads, but intros, self promotion, "please subscribe to this channel" etc (assuming creators tag their content properly). Basically it is YouTube premium for free.
An interesting thing I noticed is than initially when I used the app)(with the same YouTube profile) I would get worse video suggestions. These days they are the same in YouTube app as well as smartyoutubetv.
Another interesting observation is that if the app breaks and I start watching YouTube in their app the frequency of their ads is quite reasonable in the beginning. Few times I even thought. Hey, if they show me an ad every couple of videos that's not that bad right, I might switch back to YouTube. Then as the amount of content watched increases the amount of ads increases to the point of making it unwatchable (4 ad breaks in a 20min video, sometimes even double 40s ads one can't skip). This makes me go back.
Unfortunately, voice search is also something doesn't happen on Youtube Kids for Android TV. That greatly limits its usability, which is unfortunate, since there are many good things about it.
I'm using it in Japanese. Using the on-screen keyboard in Japanese mode, you can enter hiragana only. It doesn't do any word recognition to turn into kanji or katakana, which results in the search results being poor.
Nobody uses onscreen keyboards on Android TV for anything beyond entering a Wi-Fi password; it's a nonstarter.
YouTube Kids for Android TV ... ironically, just a toy for now ... with a well-deserved, accurate 1.4 star rating in the Play Store.
Something like, perhaps, the same standards and sense as in traditional broadcasting.
> Everything else is just a funnel
A funnel promoting pick up trucks and financial services to toddlers? Do they count that as an "impression" in the statistics that they feed the client? It seems like borderline fraud.
You can pay for YouTube premium to get rid of ads, maybe just do that? Or don't let your 3 year old kid watch YouTube in the first place, and show them something from traditional broadcasting. Before complaining about broadcasting standards, maybe first up your parenting standards.
EU providing tiny amounts of seed funding to and existing project to bootstrap a small proof of concept is entirely different from the EU trying to essentially create a tech giant.
> EU-funded project cannot succeed in tech because previous EU-funded projects have failed in tech in the past
I think the fundamental problem here is that the people that are interested and in grants and are capable of writing grant proposals are different than the people that are interested in building things. There's very little overlap. So the money goes to the people capable of writing proposals, and the people doing the work do it for free in an obscure corner of the internet.
It's sad really, but I suspect it's a side effect of the huge bureaucratical machine that is the EU. One way to make this better would be to simplify the access to grants so that technical people can do it without needing a class in "EU funding speech".
If these thinkers can find a way to remove the incentives of spammers, misinformators, and stakeholders to pollute results, that would be a great achievement. It could be seen as a big economic game, simulating these actors might allow comming up with rules to balance this game and minimize pollution.
It might be possible using moderation and reporting abuse or registering with the search engine to aid moderation and banning abusers. This will probably also take down right wing websites, porn and online gambling.
Any new open-source search option is good, but I also wish more attention was given to prior open projects like GigaBlast[0]/KBlast[1] crawlers, etc.
It hasn't escaped the wider world that quality open-source search is desirable, and it's hard to think what this new EU project brings to the table that isn't already available if others want to contribute to existing efforts. I wish the EU project the best of luck of course!
> EU-funded project cannot succeed in tech because previous EU-funded projects have failed in tech in the past (and generally government-funded project in tech are suspicious)
I'm not suspicious of all government-funded projects (back in the day my own PhD was government-funded!) but I can't help be suspicious of claims such as:
"an open European infrastructure for internet search, based on European values and jurisdiction"
and
"The project will be contributing to Europe’s digital sovereignty"
Q1: Who defines "European values"? Is that done by Qualified Majority voting or would - for instance - Hungary have a veto on any proposed definition?
Q2: Which treaties regulate "digital sovereignty"? Recalling that the 27 member states each "remain sovereign and independent"[0], is that digital sovereignty being handled in BRU, in the 27 states, or a mixture of both?
are you reading my post as being a statement that these will be the results instead of my providing an interpretation of what the original EU text means?
>Would that be the interests of the estimated 25k-30k lobbyists who work in Brussels on behalf of their corporate paymasters?
probably, as well as the various governments that exist in the EU.
You mean begging not fighting right? But why should i trust Europe more then for example Google (at least they can protect their "trade-secrets/data").
That's why People trust evil-company's more then stupid[1] Governments.
[1] never attribute to malice that which is adequately explained by stupidity.
> EU-funded project cannot succeed in tech because previous EU-funded projects have failed in tech in the past (and generally government-funded project in tech are suspicious)
Do we distinguish direct EU funding from funding by governments of EU countries? If not ASML would like a word, and I'm sure people from other member countries can come up with other examples.
Most of the negativity seems to come from the following points:
- EU-funded project cannot succeed in tech because previous EU-funded projects have failed in tech in the past (and generally government-funded project in tech are suspicious)
- Search is hard, and therefore it will fail
- The project is underfunded
Even if the points above can all be valid (though the obvious US-funded startup launched by a bunch of uni students seemed to have fared pretty well) it seems we are missing the point of this project.
The proposal is to contribute to the creation of open building blocks necessary to enable others (including private US companies) to make better search products.
Better search product are needed.
Shall we remember HN of some of the intense conversations that happened here about Google failing us:
- Google Search is Dying [1]
- Every Google result now looks like an ad [2]
- Google no longer producing high quality search results in significant categories [3]
So while, yes search is a hard topic, we should welcome initiatives aiming at improving the ground infrastructure needed to lower the barrier to entry on this subject and hope it will allow many companies to build better search products (or inspire other initiatives to contribute in similar and even more successful way)
1: https://news.ycombinator.com/item?id=30347719
2: https://news.ycombinator.com/item?id=22107823
3: https://news.ycombinator.com/item?id=29772136