Merkel may not say it directly, but a very serious risk is if Google, Facebook or Twitter's leadership decides to try to influence public affairs by hiding some things and showing others.
You may think the current leadership of those organizations wouldn't do that, but they are human beings.
* What if a powerful business partner, such as the Chinese or Indian governments or a large company, wants search results 'made more fair and balanced' in return for their business? Hollywood studios, Bloomberg, and (IIRC) Fox already have censored themselves for the Chinese government. Advertisers' influence on the news is well-known.
* What if one of the big platform's leaders feels very strongly about an issue, feels attacked, or thinks they are doing the right thing? IIRC, Jimmy Wales used his power to alter articles in Wikipedia to suit his preferences and to cover up the changes. What if a highly critical, maybe even partly false story about Zuckerberg or his wife goes viral? Will it get treated the same as other such stories?
* What happens when a big platform's leadership changes? Maybe the new people will be highly politicized. Maybe the Koch brothers or Fox/Murdoch (or Peter Thiel?) or another politicized investor will buy in - precisely to gain that influence, as many have done before.
These platforms have the responsibilities of journalists, without the institutional or professional norms and training, and with far more power than any news organization ever had. Rush Limbaugh, Fox, CNN or the NY Times each (or even together) have nothing compared to the breadth and depth of influence of Google, Facebook, and Twitter, in the U.S. and around the world.
I think everyone is missing a key point here: those algorithms are essentially incomprehensible to the layperson.
What if you demand full disclosure of the science that drives the accelerator at CERN? Well, it's "disclosed" already, just read the papers. Quantum-electrodynamics-something-something. Now what? Might as well be magic.
This is a core issue with democracy. You're asking average individuals to express a choice on matters that are orders of magnitude above their cognitive abilities.
A good point, but experts can review the algorithms and report on them to laypeople. Almost everything in the world that we understand is a result of that process, other than some things (not all) in the field we have expertise in. Nobody has time to review the source material in more than a tiny fraction of the things they know.
Just look at the "Science Must Fall" movement in South Africa.
The problem with this is simply, that if some people don't understand the science, other that ALSO don't understand it can tell them they DO understand it. Who should tell the differens, besides the people wo really understand it?
Rhodes Must Fall: protestors unhappy with having a large statue of Cecil Rhodes (founder of De Beers, "architect of apartheid", extreme white supremacist) on their campus.
Fees Must Fall: Large protests against university fee increases that used the naming convention
Shackville-TRC: advocates for alternative dispute resolution for those students who were expelled for participating in the protests.
"Science Must Fall": a hashtag made to mock one single student who said very crazy stuff in a video on youtube.
There is no "science must fall" movement documented, there was one viral story about one single person saying one single dumb thing. It went viral on Reddit in the /r/videos and /r/rage sub-reddits (those bastions of coherent discourse). It was picked up by click-bait sites who implied the existence of a movement because the angrier you are the more likely you are to click "share".
This is an excellent example of the exact problem being discussed, although I'm not convinced transparency into the algorithms is the solution here. We already know exactly why these stories were pushed out to people's feeds: this type of disinformation makes money.
Off topic: The interesting thing about that video to me was how closely it mirrored the rhetoric from the opening of the 'Three Body Problem' as the Chinese revolutionaries tried to force a college physics professor to revoke science.
Yeah but most of the time laypeople disregard the scientists and substitute their own logic anyways.. People are incredibly stubborn and set in their opinions.
I'm really hoping that yours isn't a popular viewpoint. Having access to this type of information encourages people who have the curiousity to learn about it. If I didn't have access to PubMed or arXiv or pretty much every text book ever written, I'd probably be spending my evenings playing solitaire. Instead, I get to learn about stuff I should have been learning in 8th grade.
Also, 'orders of magnitude'? I'm guessing that's hyperbole, but if not, what does that even mean?
> Merkel may not say it directly, but a very serious risk is if Google, Facebook or Twitter's leadership decides to try to influence public affairs by hiding some things and showing others.
Amusingly enough, Europe already does this through their 'right to be forgotten' laws. Criminals routinely use it to hide their past indiscretions.
People can also use internet search engines to determine when the German government is lying to them, scandalous!
> Europe already does this through their 'right to be forgotten' laws. Criminals routinely use it to hide their past indiscretions.
EU "right to be forgotten" only applies to things which are no longer relevant.
If someone's done the crime, and the time, and it's not relevant anymore, then they're done. No point in them being constantly punished for minor crimes.
Other side of the coin: is it Google/Facebook/Twitter's responsibility to eliminate spam? Because that's what a lot of output of "news" organizations is these days... one might say it's very close to propoganda. What about safe-browsing filters?
To shape your users' traffic - is that not exactly what's occurring? The question is whether they're doing it on our behalf, or for their own ends (and what happens when those two align?)
The hypothetical you pose is not nearly so cut and dry - in fact, I argue it's happening now, but it just so happens in more cases than not, that it aligns with what users want. Just take a look at the Facebook "trending" issue.
It's sort of like the very serious risk that public officials will try to influence public affairs by directly misleading, or classifying information that should be freely available because it is embarrassing or illegal, and then using their monopoly on violent coercion to keep things as they like.
I have a hard time getting riled up about the lesser actions companies may potentially take confronted with the reality of the actions governments do take.
> What if a powerful business partner, such as the Chinese or Indian governments or a large company, wants search results 'made more fair and balanced' in return for their business? Hollywood studios, Bloomberg, and (IIRC) Fox already have censored themselves for the Chinese government. Advertisers' influence on the news is well-known.
This is what _every_ business would do and actually does when it is offered a significant advantage over all potential competitors in the market. That's what lobbying is all about. Large corporations pay politicians so they create laws in order to suppress competition.
Everyone loves free markets as a consumer, but a business is never interested in the kind of competition that could put it out of existence, its employees fired and leave its founders bankrupt.
The solution here is not to create new regulations which make it even harder to do business than it already is, or to enact laws that make this kind of bribery illegal (it would continue no matter what you do) but to impose serious limitations on government power so it can't rig the system for a handful of powerful corporations that can afford engaging in this kind of bribery.
Rather than address these points individually, I think it will be more efficient to get to the heart of the issue: It seems (forgive me if I'm wrong) that the comment is based on the libertarian/Objectivist ideology that everyone is driven by greed only and that that's a desirable or necessary order of things; and that government, including regulation and law, is bad and ineffective. I don't agree with that premise, so I'd need to hear arguments that address these specific situations.
> This is what _every_ business would do and actually does
Google and the NY Times refused the Chinese government's proposals and were kicked out of that market.
> to impose serious limitations on government power so it can't rig the system for a handful of powerful corporations that can afford engaging in this kind of bribery.
I don't see how limiting government power to regulate large businesses will help; it will shift power to the large businesses, worsening the situation. I don't think Google and Facebook's success is due to government rigging the market, nor do I think their influence will now diminish if government regulatory power is reduced.
> Rather than address these points individually, I think it will be more efficient to get to the heart of the issue: It seems (forgive me if I'm wrong) that the comment is based on the libertarian/Objectivist ideology that everyone is driven by greed only and that that's a desirable or necessary order of things; and that government, including regulation and law, is bad and ineffective. I don't agree with that premise, so I'd need to hear arguments that address these specific situations.
Human advancement on every level is not driven by greed, but by rational self interest. Self interest is a necessary precondition for improving anything, otherwise what's the motivation to build for example a nice house if I don't care about the better life quality that I'll have living in it? Greed is just a subjective or collective way to say that someones self interest goes far beyond what is considered productive and rational. (if you p--- everyone off with inflated self interest you will likely have a problem rather sooner than later)
Just compare the improvements that societies made which didn't allow or limit direct self interest as the primary motivation for economic activity to those that did. There's just no doubt that life in Socialist countries was in almost every case much worse and it is even today. (USSR, pre Capitalist China, North Korea, Nazi Germany, Venezuela, Cuba and so on)
> Google and the NY Times refused the Chinese government's proposals and were kicked out of that market.
You are omitting that I wrote "significant advantage over the competition". I doubt China would _ever_ tolerate officials rigging its system in order for the NY Times to have a significant advantage over its national mainstream media. Or Google over Baidu. Most likely these companies do not even have a chance at competing with local Chinese companies anyway. An average Chinese person might desire to use an iPhone with Chinese apps on it, but reading the NY Times, c'mon.
> I don't see how limiting government power to regulate large businesses will help; it will shift power to the large businesses, worsening the situation. I don't think Google and Facebook's success is due to government rigging the market, nor do I think their influence will now diminish if government regulatory power is reduced.
I do not say that Google's and Facebook's success is due to bribing politicians, this is most certainly wrong. But large corporations do it after reaching a certain size in order to ensure that they will be able to maintain it into the future.
There would be no need to ever bribe a politician if these corporations were fully confident that they could stay dominant in the market and beat new competition. The mere fact that corporations decide to do this shows that it is often more efficient to spend resources on suppressing competition via government power than using it to innovate and improve products and services.
Who looses in all of this? The consumer (more expensive, lower quality products), the worker (lower incomes) and anyone who wants to build his own company. (high entry barriers to the market resulting in a static class of elites with less upward mobility)
Again, look at markets where government directly owned or ran almost all businesses. What was the result for the average person? How competitive were the products? These countries usually allowed and desired to export while suppressing imports, but for some reason all the Capitalist societies wanted to buy was natural resources, not finished products. Buy Soviet Lada cars? Who would have bought that when there's Mercedes, Audi, BMW, VW and so on? (I'm in a German country so that's what we drive here)
Of course there were a few cases were largely Socialist countries did work, but these were to my knowledge always edge cases where they for example have a very small population and enormous natural resources (like oil)
> Libertarian/Objectivist ideology
Yes, something along these lines. I desire to live in a society that offers as many freedom to every one of its members as possible, though I recognise that there must be limits too. The Socialist on the other hand wants to mold society in some way or another in order to achieve or reach some kind of higher or better form of society. This is always done by the use of force and I reject that. If you try to convince me to better myself - fine, but passing laws that threaten fines (robbing me of my resources) or prison is not okay with me.
Unfortunately for you, reality itself and events that actually occurred have a pro-Clinton bias. Trump did actually shit all over himself during the debates. Google is merely reflecting reality in this case.
Totally agree. It's very easy for Google to tweak their algorithm little by little to affect public opinion over time. I think if the change happens slowly enough; nobody, would notice.
I doubt that Google would tweak their algorithm in such a way that it would work against their interests (if this happened, I bet they would just roll back the change) - That alone gives Google an advantage over everybody else.
It's a bit like how some manufacturers were able to slowly lower the quality of their products over time and still retain or improve market share because people just didn't notice...
zuck shadowed no tech execs early on his ceo career. he followed though the ceo of the wall street journal because he viewed fb, this is early days too like 2005, as more of a news organization. he knows what's up. so to know the future of google/facebook etc. just read about about the previous tumultuous 100+-year history of the media particularly that beginning part with william randolph hearst.
Many tech engineers are extremely principled, to the point where it would be impossible to pull something like that off without someone finding out about it and blowing the whistle.
Many very bad things have happened in large organizations, and have persisted for years or decades without someone blowing the whistle. The Catholic Church, intelligence agencies, and many police forces come to mind.
I don't think engineers are more principled than others, and thinking we are exceptional is, IMHO, the number one way to expose ourselves to these human frailties.
I think oh_sigh might have meant to say "the rate of extremely principled individuals seems higher in geeks vs non-geeks" -- which I think is true, even if the average rate of principledness is around the same for both groups.
So I'd like to add to your rebuttal that extremely principled people are also capable of awful actions. All it takes is a good theory.
I'd agree that "principledness" seems unusually high in geeks, except that those principles are often bad ones which lead to bad outcomes. Which is a lot more dangerous than a hypothetical average amoral schlub.
Being extremely principled is exactly why many of these companies and/or their engineers might be tempted to put their thumb on the scales. After all, this election or this issue or this whatever is just so super important that it would be immoral not to take a stand...
I mean, just think about how many people have been saying literally that for this election! If anything, the strongly principled are the ones I'd worry about the most.
Find me any significant system in a company like facebook or google where only 1 person or even only 3 or 4 can sneak something in without anyone else finding out about it.
Extremely principled about who should win the election.
I don't want to turn this discussion into the presidential mess, but it's no secret that virtually almost everyone working for a particular search engine, from employee to executive support one particular candidate and not the other.
I work for a particular search engine and I support the other candidate.
A small group of people here are very vocal about who they support, but probably 90+% of people never mention politics and don't seem to like to talk about it at work.
When organizations do something highly questionable it usually happens in teams where people have been selected, through time, to accept the ethical or legal consequences.
I guess I'll say I only know the google model of things, where pretty much 99.9% of the codebase is open to FTEs. It would be very very difficult to slip anything into the code there without someone catching on and causing a shitstorm.
It is an interesting and valid point, but hasn't this been always the case ?
Media distort perception almost by definition, the difference is that now there are new actors and technologies, which are mostly outside of governments' influence, unlike traditional newspapers and television.
I'm not saying that it is an improvement upon the old status-quo, but let's not pretend that before it was much different.
No media organizations in history have had anything approaching the influence of Google and Facebook, unless you count state-controlled media in totalitarian countries.
> the difference is that now there are new actors and technologies, which are mostly outside of governments' influence, unlike traditional newspapers and television.
I don't see how German or U.S. news media are more under government influence than Google and Facebook.
Chomsky would strongly disagree with you - he is of the opinion that market pressures, and more importantly, access to people of power, turns mass media into an echo chamber, parroting narratives that do not diverge very far from the orthodoxy.
It's a form of soft influence - nobody needs to walk into a CNN office with a baseball bat in order to keep them on the straight-and-narrow. CNN does a good enough job policing itself.
It's an intriguing theory and worth bringing up, but it's also a conspiracy theory in that it's impossible to substantiate or disprove, which is a little too convenient.
I'm sure it happens to some degree, but perhaps Chomsky is unhappy that most people disagree with his views and attributing it to some hidden force.
> I don't see how German or U.S. news media are more under government influence than Google and Facebook.
Remember it was only a couple of decades ago that the CIA was caught mass infiltrating news networks (e.g. Armstrong, etc) and that since then a number of intelligence officials, family members and military allies have filled their board positions and editorial positions.
There’s also the ‘trade’ for access to off-the-record context and stories, suggestions for story edits by the intelligence community on everything related to national security (e.g. Dilanian), the collaboration between the intelligence community and journalists to break stories important for policy positions even if they are specifically disinformation (e.g. Judith Miller and justification to invade Iraq), broadcasting influence centered in the BBG (for instance the hundreds of advertisements in US news supporting the Iraq war that were out of the Kenneth Tomlinson led BBG), the membership of the National Security personnel on the curators of the domestic and international news wire service, and the huge amount of money from US government press centers to news organizations to report on specific items (‘free press’ means press on whatever you can get paid for), and media imbedding rules that prevent journalists from reporting freely on war (rules that we would criticize where they practiced by any other nation).
Josh Earnest, the Press Secretary for President Obama, has bragged about creating echo chambers in the domestic media.
Propaganda protections are widely overestimated in this country. When you look at the legislation and you look at the practice you begin to understand how the law is interpreted and practiced.
Yes and no. Yes, it has always been the case - but the trust we put into different information sources is different. Most people are pretty sceptical of traditional media. But I'd claim they are less sceptical of search engines - because they provide an illusion of agency and choice: You decided to research a particular topic and you "found" a number of different sources, none of which are associated with Google. That appears a lot more trustworthy than some opinion piece in a newspaper.
Perception distorts perception. The problem of media compounds the problem that individuals create their own perceptions out of a combination of incoming stimuli, prior experience and emotion. Science and philosophy (and other metaphysics like religion) all dispute the implied corollary of one objective reality, and as far as I know they also fail to prove any such happenstance.
I don't mean to deny your point, instead try to widen it out with an aim to finding a more useful general solution.
Broadly speaking, the search engines aren't disclosing these algorithms because we know then people will abuse them. Disclosing them publicly may have the paradoxical effect of introducing even more distortion into the system.
The only solution I can see is just ensuring there's more competition, and even that isn't a perfect one.
yes -- for most topics, the curated information in a wikipedia article is better than the first page of G results.
This is partly because curation still works in 2016, but also because G results are easier to monetize. A significant portion of web content is some form of clickbait, i.e. ad-supported pages with no new information on their topic.
For now, viral content seems like a bigger threat to search results than abuse by the index operators. (Though that can change in a hurry; some studies have shown that search rankings can be used to influence get-out-the-vote numbers).
But again, you could also argue that there would be fewer Linux exploits if it were closed source. And Linux exploits have definitely caused loss of revenue for web hosting companies, and preventing that kind of abuse does carry weight too.
With search engines, there's two things. If the core goal of a search engine is to link to ads, (or pages with ads) then bad-actors will be incentivized because there's money to be made. If the core goal was simply to link to content then the incentive goes away, and so does the problem.
> If the core goal was simply to link to content then the incentive goes away, and so does the problem.
No, it doesn't, because the core goal of even a perfectly non-profit search engine is to sort content by relevance. That's ultimately a general-AI-complete problem that would require your computer to read your mind in order to solve it perfectly. So necessarily, a search engine is using some heuristics to approximate relevance, and those heuristics are open to gaming. Thus the cat-and-mouse game between search engines and SEO people.
There's way too much content to just link to all of it.
The SEO people are only involved because google links to ads, or pages with ads. For the sake of argument lets say google opensources their engine. One could create a forked search portal that never linked to ads or pages with ads. The motivation to game such an engine would be greatly reduced IMO.
I disagree. Outside the startup world, a lot of SEO is done simply to drive traffic to you so that you have chance to convert some of it into paying customers. All the small businesses that pay SEO companies to spam the shit out of the Internet don't do it for ad revenue, they do it just for organic traffic.
This motivation won't go away as long as people can use the Web to make money, so we're stuck with the pressure to game search results.
Maybe I should have differentiated between link-farm type SEO and web marketing. The former is more likely to cause people to abuse the system because simply getting people to visit your website or click a link provides monetary compensation. If you drove traffic to your website I could make a hand-wavy free market argument that if your product wasn't good, it wouldn't sell and you wouldn't have money to pay someone to drive traffic to your website. I fully accept that its my own opinion.
Would you want the non-profit search engine to be on par with Google for quality? Google's storage just for the index of web pages is about 100 petabytes.[1][2] To compare, Wikipedia's storage is only ~50 TB[3] and it's mostly static content with no heavy cpu constantly processing the data. Relatively speaking, Wikipedia's operation is tiny and yet they're regularly running banner ads for donations to keep the lights on.
Copying an open-source search engine algorithm from github isn't enough to get a usuable search tool. And buying some 8 terabyte hard drives from Amazon to run a RAID setup in the garage won't store enough data to feed the algorithm.
I believe you could drop 99 of those 100 petabytes and your average user wouldn't notice. The web is full of crap. I don't think you have to match Google's size from the start to be able to compete.
That was my thought as well. Especially if in some way this could run as a "personal" search engine. So I could fine tune it to avoid crawling and indexing certain sites - I really don't need to see anything from CNN, HuffPo, etc. If I have that urge, I can always Google it.
>TeMPOraL wrote: I believe you could drop 99 of those 100 petabytes and your average user wouldn't notice.
A general purpose search engine would need that 99 petabytes of bad webpages to help machine learning algorithms classify the new and unknown web content as good or bad.
>if in some way this could run as a "personal" search engine. So I could fine tune it to avoid crawling and indexing certain sites
What you want sounds more like a "whitelist" of good sites to archive and a blacklist of sites to avoid. With the smaller storage requirements of the whitelist content, build a limited inverted index[1]. I agree that would be useful for a lot of personal uses but it's not really a homemade version of Google. Your method requires post-hoc reasoning and curation. Google's machine learning algorithm can make intelligent rankings on new content of new websites that don't exist yet.
Maybe not on par but close - I was expecting limitations like that, didn't realize it'd be in the petabytes! I wasn't saying or expecting that it could be done on the cheap. Just trying to think of ways to get away from the filter bubble and ad-based businesses.
I was just daydreaming, without first evaluating technical limits, that there could be a distributed search engine that runs on donated CPU/storage like those SETI and genome projects.
The Internet Archive has less ~200 to 400 billion pages. Google has 60+ trillion pages.
The IA also doesn't spend money on constantly running map-reduce[1] jobs 24-by-7 which requires a datacenter of cpus and megawatts of electricity. By extension, they don't pay for a small army of programmers to write the ranking algorithms and tune the results.
So on their shoestring nonprofit budget of $12 million a year, you won't be able to get good results from search queries.
E.g. type "javascript" in the IA search box and the first 10 results are junk.[2] At the top is a Javascript Bible book from 2010 that's rated 3 out of 5 stars on Amazon.
Compare IA with the "javascript" results from google.com[3].
It will take a lot more money than $12 million a year[4] to upgrade IA to be similar quality to Google. The increased costs would very likely exceed the financial support of their most generous donors.
Talk about feeding the troll ...
Your argument is invalid, simply because google didn't become big because they had more money than others, but because the technology was an advantage.
Sure, expenses are a limiting factor along the way, but you use anecdotal evidence to make a point of that. Look at traffic instead, is google also single handedly stemming the whole infrastructure?
The question was about other search engines, non profit at that, and in doubt of that, about the possibility. Whether or not it would be expensive and whether or not the wikimedia foundation uses a lot of their money for meet ups and the like, there are possibly other limiting factors, that might ease the financial problem when overcome.
Is "technology advantage" another way of saying their PageRank algorithm of mathematically iterating on a linear algebra problem was superior to other approaches from Lycos/Excite/Yahoo? Well, yes.
But that algorithm has to run on thousands of CPUs which require lots of electricity and expensive programmers to program it. The algorithm has to be expressed on real-world things that cost lots of money. Google got $25 million of VC money in 1999. 18 months later, they figured out Adwords and that brought in enough money to self-fund expansion (buy more computers and harddrives). The IPO in 2004 brought another $1.9 billion to pay for datacenters. The superior algorithm must combine with expensive massive hardware scale to deliver quality results.
>The question was about other search engines, non profit at that
If you read jkaunisv1's comment more carefully, he wanted the homemade search quality to be "close" to Google's quality. You can't achieve that on a nonprofit budget of $12 million a year. (See my links for "javascript" to get an idea of what $12 million buys you.) To be fair to IA, their main mission is archiving and not cutting-edge search algorithm quality.
>With search engines, [...] If the core goal was simply to link to content then the incentive goes away, and so does the problem.
Why would the incentives go away for bad actors to game the ranking algorithms so that their bad content rises to the top of results?
Your chain of logic doesn't make sense. E.g. Since the core goal of SMTP is exchanging electronic mail -- it means the incentive for for advertising (SPAM) goes away and the problem solves itself?!?
Why would spammers and bad actors let the idealized goal of technology stop them from abusing it?
>Why would the incentives go away for bad actors to game the ranking algorithms so that their bad content rises to the top of results?
Then you're talking about bad-actors who are not motivated strictly by financial gain, but by the successful spreading of their bad content. That is not really a problem specific to search engines. It applies to any medium that has the ability to disseminate information.
> E.g. Since the core goal of SMTP is exchanging electronic mail -- it means the incentive for for advertising (SPAM) goes away and the problem solves itself?!?
That is not comparing like to like. Unlike mail servers, google plays a very active role in choosing what kind of content it indexes. My point was that if they chose to simply index information and never link to ads or pages with ads, then the incentive to game the results is greatly reduced. (For e.g. - Google Scholar.)
>bad-actors who are not motivated strictly by financial gain, but by the successful spreading of their bad content.
The "bad content" showing up on the top of page 1 of the search results is directly related to financial gain. That's why the SEO industry exists!
>if they chose to simply index information and never link to ads or pages with ads, then the incentive to game the results is greatly reduced.
The flaw in your logic is that it's the ads that's the primary motivation for bad actors to push their unwanted pages to the top.
Let's say I'm a bad author that wrote a bad book about "traveling to Paris". I would want my blog page about me and my book to be at the top of search results when you type "Paris" in the search box. It doesn't matter what ads are showing in the side panel. It also doesn't matter if ads didn't exist at all because every Google user would pay a $9.99/month subscription. Either way, I still want my webpage at the top so that some percentage of search users click on my (non-ad) link and buy my sightseeing book.
>The flaw in your logic is that it's the ads that's the primary motivation for bad actors to push their unwanted pages to the top.
Yes, that is my opinion.
> I would want my blog page about me and my book to be at the top of search results when you type "Paris" in the search box. It doesn't matter what ads are showing in the side panel. It also doesn't matter if ads didn't exist at all because every Google user would pay a $9.99/month subscription. Either way, I still want my webpage at the top so that some percentage of search users click on my (non-ad) link and buy my sightseeing book.
Okay, but that is simply marketing. You are not assured of any monetary compensation if someone visits your website or clicks on a link. Its like 'buying' influence in a bookstore to make sure your book is displayed prominently. I was primarily thinking about the SEO surrounding link farms and other similarly abhorrent web sites.
>Okay, but that is simply marketing. You are not assured of any monetary compensation
It doesn't matter that I'm not 100% assured of a sale.
What matters is that I have used up the finite space of pixels with a link to my bad webpage. Add in other bad actors like me gaming the ranking system and now the first page of results is full of spammy web pages. The "good" links such as a wikipedia article about Paris is completely pushed off the first page. If we get 0.01% of eyeballs to our "bad" links converted as sales, that's still better than zero.
As a user of Google and Bing search engines, I do not want any government to force them to publish their algorithms. It will make the search results worse. Keeping the algorithms and heuristics a secret is a valid way to fight abusive gaming of the system.
The opaque strategy for search rankings is not the same issue as a closed-source encryption algorithm with a backdoor.
>Where did you get the 0.01% from ? That sounds like a rather high number.
Why are you distracted by that 0.01%? Whether it's 0.001% or 0.00001%, it's still higher than zero. It still drives the incentives to push bad pages to the top of search results.
>, there is no evidence that this is true.
The evidence is Google's ongoing evolution of algorithms from the 1998 PageRank paper[1]... to Panda... to Penguin... to Hummingbird. All that constant rewriting is to stay ahead of the abusers gaming the search algorithms.
The link farms were created by spammers based on the information about Pagerank and using its link-weighting to the spammer's advantage. The public knowledge of how PageRank works allowed spammers to make search results worse. There are many examples[2] of of link spam that is not motivated by the "ads" in the right side of the page. What you call "web marketing" is often bad pages made visible by abusive SEO techniques.
Panda was a response to this gaming by analyzing extra signals to penalize link farms. Abusers continued to game Panda's revised algorithms and Google responded with Penguin.[3] Just stop and think deeply about why we can't just use Google's original 1998 PageRank algorithm unchanged in 2016.
At this point, making all of the algorithms and weights of Hummingbird public will only give the spammers the necessary information to make the search results worse for the rest of us.
If your belief is true, why does email spam exist? There is no way to way to tell if they were opened (especially with modern email clients that will not load external links), so there is no direct advertising happening.
Its the argument that the source must be hidden or else people will release exploits for it. e.g. The fact that Linux is open source has caused people to release exploits for it.
With exploits, you can easily fix them. The problem with this sort of algorithm is that you have people directly competing, and there's no objective measure of site quality. The system has to use what it can measure, and this means it's gamable. Security through obscurity sucks, but it's better than having nothing at all.
There's a reason the US military put a psychological operations center next to the Googleplex.
Merkle is absolutely on the right track. It's been shown study after study and history after history that information providers - be they news, aggregators or indexers - have an incredible amount of power over large populations.
I'm just afraid that what Merkle means is that they could 'distort perception' away from the perception she'd like to see people have and that this may justify working with these companies to "undistort perception".
The 63th Regional Support Command (linked above) hosts the 12th and 14th Psychological Operations Battalions out of the 7th Psychological Operations Group.
I'm reminded of the argument in 'Amusing Ourselves to Death' that the predominant medium shapes the culture. It presents the argument as a juxtaposition between a pre-telegraph textual culture and a modern-day visual one: The textual culture encourages slow reasoning, contemplation and logic, the visual culture encourages sound bytes and de-emphasizes contemplation and reasoning; a TV show simply won't give you enough time for thought before it moves to the next idea.
I think the point I'm trying to make is that all media has a "selection algorithm" that distorts reality - TV and the news are great examples of this.
There was a pretty good essay in the German computer magazine c't sometime in the early 2000nds, were the author argued that a computer is a meta-medium, that is a computer can emulate almost arbitrary types of media. A computer can work as a TV or as a game console or as a textual medium.
From this point of view, it is no longer the case that there is an inherent selection algorithm that is generated by what works well in the medium, for example a game show just would not work as a news paper article, but that one can decide which advantages and disadvantages a computer has. And so it is more or less the first time, that one can decide which selection algorithm is the relevant one.
I hear that line of reasoning frequently and Im not sure it has a leg to stand on. Telegrams by their nature are going to be terse and infrequent certainly not a ponderous essay you can mull over. And are we seriously arguing that in the, comparatively, "larger" world of the early 20th century where many communities could be completely isolated from differing ideas there was some how greater access to a wider variety of information?
My personal hypothesis (although Im not as familiar with more rigorous investigation into the subject) is reactionary culture in the "first world" is less to do with content accessibility and more to do with availability of time as compared to the signal to noise ratio of our information sources. It's not that people are becoming less informed than the past, just that we have many more tools at our disposal to find conflicting information.
It's certainly a dangerous thing to put on the rose colored glasses and look to the past.
>Telegrams by their nature are going to be terse and infrequent certainly not a ponderous essay you can mull over.
Which is beside the point, since the parent talks of a pre-telegraph terse text culture, not a telegraph one.
>And are we seriously arguing that in the, comparatively, "larger" world of the early 20th century where many communities could be completely isolated from differing ideas there was some how greater access to a wider variety of information?
No, the idea is that there was deeper involvement into the available information -- instead of always skimming it, which ends up not delving enough into anything.
>It's certainly a dangerous thing to put on the rose colored glasses and look to the past.
It's even more dangerous to think of history as a kind of monotonic progress across all sectors (as opposed to just the technological one).
Merkel seems to be conflating the issue of search engine algorithms with social network algorithms. Search engine algorithms do their best to find what you are probably looking for -- it is a self-selecting filter bubble and they are working as intended.
Additionally, there are numerous search engines, unlike Facebook and Twitter which have effective monopolies over social media news sites.
In the case of social networks, Zuckerberg is 100% on the money on this one. Anecdotally, I've noticed my Twitter and Facebook feeds are extremely diverse in opinion and philosophy. The articles linked to and discussions had on each of my home pages are wildly different than what I would find if I were only searching through the news sites I habitually visit.
I think Google's personalization of search is good for filtering out noise in most day-to-day searching.
If an individual finds personalization of search to be a barrier to finding diversified news, they could always use incognito and search among many different search engines for the most diversified results.
Merkel's solution to the problem is to add another roadbump via government intervention. The real solution is already in the users' hands.
> If an individual finds personalization of search to be a barrier to finding diversified news, they could always use incognito and search among many different search engines
Few people will do those things. Most people do not understand Icognito and won't look at the second page of search results, much less use multiple search engines. I wonder how many are even aware of other search engines.
The reality is that the search engines' personalization algorithms determine what users see.
The reality is also that the quality of users' search results is directly correlated to the skill of the searcher and consequently the quality of their searching process.
You look at it as a problem for the user - the user isn't getting the diversity of opinion they want to get.
Merkel looks at it as a problem for society - people aren't getting the diversity of opinion they should get.
If I wanted diversity of opinion, I would go to Breitbart, but I don't want to. Maybe it would broaden my perspective to do so, but I can't be arsed. I've been there once, but it was mostly an exercise in social anthropology.
She isn't, the Guardian is. If you read the article closely you'll realise they are the only ones who speak specifically about search engines. What is quoted about her speech is all about "algorithms" and "internet platforms".
If only I had a dime for each time a politician tried to explain their own downfall through supposed distortions of reality, or people's inability to swallow a particular narrative.
This belief that there is an always an understandable algorithm you could look at is an interesting one. More and more decisions are being made by systems that are opaque even to their designers, who can only shrug and point to the training data. Among other things, it's a very effective way to disclaim responsibility.
What is actually distorting perception is pressuring Facebook to delete free speech without any legal basis, because the government defines it as "hate speech". No one knows the exact definition of "hate speech" though. Political correctness to the extreme.
Please do not assume that US-American ideals are universal.
Germany has a complicated history, which it has and will always struggle with. Restricting hate speech and symbols is completely legal and also seen as an important and good thing. The modern German constitution was built especially to defend its democracy even against a majority hostile towards it (Artikel 20). Germany has learned its own lessons from history.
I'm actually german and I do not agree with those laws at all. I'm not even affected by them, I just happen to think that free and unrestricted speech is most important and should apply to everyone and anyone, regardless of what is being said. Private law is a different matter though.
If the speech Facebook is forced to delete is illegal, there must be criminal investigation, by law. So it can't be illegal, which means the government is actually suppressing freedom of speech.
> Restricting hate speech and symbols is completely legal
"But it is legal" is not exactly a helpful response to criticism of a law.
> If the speech Facebook is forced to delete is illegal, there must be criminal investigation, by law.
I agree with that. Facebook should not be the instance deciding what is to be deleted or not. The german judicial system should be involved and offenders be prosecuted within the judicial system.
We tried shame and humiliation as good things and governance tools in Germany before, between 1919 and 1933. We know how well that worked the first time. Yet, it's become the governance tool of choice there again.
For years, whenever I talk to German friends, the fear and timidity about discussing the German past from them is palpable. They just do not want to discuss it. All you have to ask them is, "So what do you think about World War II?" and you will see them start to sputter, slip into a kind of trance, and recite a spiel about how Germany did some very bad things. Even outside their country. They can't even talk about what is happening now. When I was in Berlin this year, in a very nice place in a very nice part of town, my host told me to make sure to lock up because there had been a steep rise in break ins and robberies in that part of town. I could tell he wanted to say "because of the migrants", but he caught himself and held his tongue. To point out the emperor's lack of clothes is verboten in Deutschland.
And now, again, we see the stirrings of strong, proud nationalist elements in response to all this, as what were previously fringe parties gain steam yelling about the Lugenpresse, and more and more Germans listen.
Hasn't this all happened before?
I'm not sure when political parties and other interested parties will figure out that shame and humiliation are terrible governance tools that only build up resentment and cause reactance in humans. Because they sure haven't figured this one out yet.
He wasn't assuming that American freedom of expression is universal, but we can assume he was already aware of your "explanation" of why Germany is the way it is.
Explanations of why doesn't mean it's a good thing.
Essentially, the population at large decides that a thing should be censored for the greater good, but it can't be exactly defined, yet a consensus can usually be reached on any given example.
The speech that is illegal in Germany is the type that encourages genocide of particular ethnic/religious groups.
I, for one, believe that we should throw all members of your ethnic group into gas chambers and incinerate your bodies. I'm legally allowed to say this because I live in the USA.
Freedom of expression or privacy, only one can be protected at time. If you want to say whatever you want to say, you should acknowledge responsibility.
When I want to read a contrary view, I don't subscribe to a new magazine or type in the address of a blog I've never seen. How would I know which one to look at? So I Google it.
I don't want the algorithms released to the public where spammers can read them. That won't help diversity, I'll just get articles from the best spammers.
I wonder if Merkel already knows better but thinks this statement helps her politically somehow.
>The term, coined by activist Eli Pariser, refers to the algorithms on social media sites and search engines that show users information based on things they have previously liked or searched for. As a result, many people who follow news in social media are exposed to a world view similar to their own, without being shown opposing views.
People happily segment themselves by only watching newsertainment they agree with. A lecture series on search algorithms won't change a thing.
Are people really interested in seeing how the sausage is made or just politicians?
Sure, the average slightly-more-technical-than-thou person gripes about Google and Facebook being biased, and in some cases they're right, but ... would they actually be capable of discerning the truth correctly?
Or would they just be more overloaded with more information?
Angela Merkel is extremely upset that the population management techniques she is familiar with from East Germany are no longer effective when people are able to search out opposing points of view. That's why she's leaning so heavily on tech companies to implement censorship apparatuses, with the threat of "audits" and "transparency" monitoring should they not comply.
The "population management techniques" are not working on East Germans, because they are inoculated (see Saxony). Wessies do not have that experience and cannot read between lines yet.
I'm all for it, but only because I'd love to poke around in the google-brain. For the problem of filter bubbles, I don't really see how it would actually help?
I mean, I know Facebook will avoid showing me articles from breitbart, and google knows that 'python' is a mostly a language for me, and not a snake. Both companies have said that, repeatedly. Opening the algorithm would help if there were a secret conspiracy to influence people via search ranking, but nobody is seriously alleging that.
So that argument is ok to protect the integrity of your SERPS, but people get all red in the face when it's used for safekeeping the nuclear launch codes?
I don't believe gp was talking about security by obscurity[1] and it would be wrong to frame it that way.
Instead, it's ranking and weighting by obscurity and it's a valid position to prevent gaming by bad actors. Statistical weighting is not the same topic as security such as closed-source cryptography.
Since Google uses RankBrain now as a signal which is trained on the web itself, how would the outputs of this be described as an algorithm? I think politicians are imagining that search is like some step by step recipe when it's incredibly complex now.
I don't think even Google engineers fully understand DNN results on such a large corpus.
I see, then you seem to be a person who makes wild claims and pastes random links in their comments unrelated to anything. In that case its easier to just ignore you. Thanks for clarifying that.
If Angela Merkel had it her way with search engine algorithms, every news source within the first 50 pages of Google.com would paint Germany in a positive-light (especially her barbaric open-door immigration policy)
Because of the internet people are finally hearing the news without a filter. They are no longer getting news they don't care about or don't want to hear.
So they are actually controlling the politicians as they want. Not as the old news/system told them to.
This is what is actually scary. Having the peoples voices heard isn't the utopia we thought it would be.
I don't think it's as clear as you're making it out to be. Any kind of targeted content grooming is inevitably going to form a bubble of advertising and news that, according to the algorithm, fits your profile the best. Is that not altering the way people perceive the world? Intentional or not, it's still self reinforcing; is that not creating a sort of bias?
I take it that in Germany when I go to the news stand I hand over my money and the shopkeeper gives me a randomly selected newspaper? Or do I still get to distort my perception by choosing my favorite newspaper? Are the daily editorial meetings of German newspapers subject to government oversight?
In the US, I would expect this to run afoul of the First Amendment. The freedom of speech is protected, and you cannot be compelled to explain how you decided to say what you said.
I ask a dude standing on a corner to give me directions to a place. Now,the dude is supposed to publish his logic as to how he arrived at those steps in the directions?
Google is not supposed to respond. The demand is addressed towards the public, not Google. It's kind of a form of pressure, they are showing they are ready to escalate and are hoping later on Google would be more willing to cooperate with the government and so on.
> Merkel and the EU army of lawyers actually pulls this off
> Google and everybody else release algorithms
> SEO "experts" and other blackhats now have the holy grail to good ranking, defeating the purpose of said algorithm
In reality, I am not really so sure that Angela Merkel in particular would like Google to expose how they attempt to distort reality, history or politics...
She should have demanded measures against "terrorism" and "pedophiles", like other governments. People seem to be ok with censorship presented this way.
So, politicians have trouble influencing people through social media, search engines and decided to openly demand influence there. It's good, it means they were not able to persuade them privately.
Yeah, it's always the public at large, or companies that need to be "transparent", but when it comes to the government it's "do as I say, not as I do".
Angela Merkel is right for once here, these companies can and do influence public perception and even elections.
This is seen in the current US election where all except for Facebook quite overtly try to suppress the conservative viewpoint.
But my big issue with what she is stating (aside from the refugee invite to the EU) is that she wants governments to control what kind of message is put out.
With private control and ownership we still have at least some media that presents an alternative viewpoint to the current left leaning mainstream. (I'm talking about big corporations)
With government control it could easily be 100% percent biased in one direction. It's bad enough already but this would be even worse.
People end up more and more segmented into their filter bubbles. This has led to extreme behaviour during the US elections already.
We really really don’t want to repeat that horrible situation with the federal election in Germany 2017. The question is how we can prevent that.
________________________
My personal suggestion would be to require Google to only use non-personalized ranking for all news and political articles, and to require Facebook to add fact-checking previews to every single article posted discussing a topic on which a fact-check exists.
So that if someone posts a Trump tweet on Facebook, in the preview it’d also display a short fact-check from both sides of the political spectrum, and any longer article would also always be presented together with an opposite point of view.
The traditional media have been required to present multiple points of view on topics, and to fact-check political statements for ages, such a solution might be useful for the modern media, too.
________________________
But to go back to the start: If there’s one thing we don’t want to repeat, it’s Trump, or the AfD, or any other populists propelled to power through misinformation and propaganda.
Perhaps unintentional, but your examples are extremely biased.
More than one politician has distorted reality and played fast and loose (to be charitable) with facts.
Moreover, who checks the fact checkers? Politifact and other sites aren't unbiased themselves, after all, and we certainly have seen that the news media in America are easily manipulated (c.f. the tons of free media Trump gets and the collusion explicit in email dumps from the DNC hack).
We don't have high quality, (mostly) neutral fact checkers in this country. That's a big contributing factor to the state of our current election, where the highest office is contested by a bigoted, stupid clown and a corrupt, deceitful and likely criminal woman, both of whom are likely to engage in terrible and aggressive foreign wars, while just about anything down-ticket of these two is ignored unless it intersects with the Presidential race or involves some completely irrelevant and salacious sex scandal.
> We don't have high quality, (mostly) neutral fact checkers in this country.
I disagree. The leading news sources, including the NY Times, Washington Post, and others, do a very good job. In case you think the Times is biased, remember that they broke the Clinton email server story.
One data point isn't sufficient to make your case, in my view. And the bias isn't on a Dem vs. Rep axis but rather on an axis of class, especially as regards access to socially, politically and financially well connected Elites.
So that if someone posts a Trump tweet on Facebook, in the preview it’d also display a short fact-check from both sides
I don't even know what a partisan fact check is. If the intent is to fact check, there can't be multiple interpretations of it. A ranking of how truthful it is, which parts of a statement are accurately representing reality, like politifact's truth-o-meter, would be useful. Partisan fact checking is how you get things like "clean coal".
Fact checking, however, is a fight against entropy. It is easier to lie or misrepresent, even inadvertently, than it is to be fully informed enough to be always say fully, 100% true statements. The liers are not lying for lying's sake, they have an agenda, and they'd just as soon tell the truth if it would serve their purposes.
"Fact-checking" doesn't work, because now you've got a single exploitable point of failure. All your ideological cronies have to do is get control of the fact-checking department.
You may think the current leadership of those organizations wouldn't do that, but they are human beings.
* What if a powerful business partner, such as the Chinese or Indian governments or a large company, wants search results 'made more fair and balanced' in return for their business? Hollywood studios, Bloomberg, and (IIRC) Fox already have censored themselves for the Chinese government. Advertisers' influence on the news is well-known.
* What if one of the big platform's leaders feels very strongly about an issue, feels attacked, or thinks they are doing the right thing? IIRC, Jimmy Wales used his power to alter articles in Wikipedia to suit his preferences and to cover up the changes. What if a highly critical, maybe even partly false story about Zuckerberg or his wife goes viral? Will it get treated the same as other such stories?
* What happens when a big platform's leadership changes? Maybe the new people will be highly politicized. Maybe the Koch brothers or Fox/Murdoch (or Peter Thiel?) or another politicized investor will buy in - precisely to gain that influence, as many have done before.
These platforms have the responsibilities of journalists, without the institutional or professional norms and training, and with far more power than any news organization ever had. Rush Limbaugh, Fox, CNN or the NY Times each (or even together) have nothing compared to the breadth and depth of influence of Google, Facebook, and Twitter, in the U.S. and around the world.