As someone who has worked in the media industry, I can tell you that the problem with unbiased news goes much deeper than simply presenting facts objectively. The very first step mentioned in your breakdown - scraping the latest headlines from publicly available news sites - is inherently biased, because it relies on a small selection of sources to determine what is considered newsworthy. But by whom? In reality, editors play a crucial role in shaping the news by deciding which events to cover and which to ignore, based on their own biases and interests.
From my own experience, I can tell you that the most important news stories are often the ones that go unreported/underreported by most news sites. Unfortunately, there are also many insidious methods used to shape public opinion, such as distraction etc. But it all starts with the decisions made by editors about WHAT YOU, as a reader, should SEE and CARE about. So if we truly want to achieve unbiased news, we need to start by reevaluating the very definition of what is news.
I find the closest to an unbiased news source is Reuters.
I removed all other news apps from my phone and only leave the Reuters news app installed. [1]
I never read the news anymore.
The combination of the uncurated deluge of articles about stuff I don't know anything about (and realize I am not going to ever really know anything about) and them not really poking my buttons with editorialized headlines makes me just ignore it and come here more often instead.
Edit: I fathom its because the Reuters business model is fundamentally different - they are just trying to supply as many articles for use as raw base material factoids to all types of media organizations who can then add their own "secret sauce/value" by curating which articles their audience "sees" and handle the "interpretation" of the factoids (i.e. insert the editorial opinion with headlines, quips for their audiences will appreciate). That said - I do recall reading about some instances that Reuters is not perfect (article rushed to press and not entirely factually correct).
Reuters isn't so much a source of news, it's a (top level) news aggregator.
A source would be an experiencer telling their story about the events considered new, i.e. unexpected.
News are updates of our collective model of the world.
Human communication has the aspects of content, sentiment and style. People look at style first, as they take it as an indicator for the social standing of the speaker. Sentiment is a close second, as it tells you about the attitude that speaker has towards the reported events. Content lastly is just the events themselves as they (reportedly) happened.
People find that content part hard to follow on its own, mostly because they usually don't really know the context anyway. Focusing on it as supposedly important "unbiased" "true" news is a gross misunderstanding.
Like yourself, people generally have a very incomplete idea of the world they live in.
Like you, they don't seem to take much interest in it, as they view it as "not relevant to their lives".
That is an absurd statement to make, if one has no clue about that part of the world to begin with, like you observed.
> “ In 1969, Reuters needed money to further expand in the Middle East and Western powers such as Britain wanted to bolster their influence against the Soviet Union by expanding news services across the world, the documents showed.
The secret government financing of Reuters - as set out in the documents - amounted to 245,000 pounds ($317,838 at current exchange rates) per year before 1969 but then reduced to 100,000 pounds per year in 1969-1970 and nothing in 1972-1973.”
I don’t grasp what the article’s writer meant by “current exchange rates” and “before 1969” here, but in any event it sounds like only a few million real constant 2023 dollar equivalents, total, were involved, which is less like a secret think tank and more like travel and bureau support, which is basically what is being admitted to.
> I find the closest to an unbiased news source is Reuters.
Maybe for American news it is unbiased across conservatives / liberals, however from a non-American point of view Reuters is very VERY much biased towards US / CIA (even though it is incorporated in England).
Underreporting on violence in certain communities vs others is biased. Who determines what is uncommon vs common? Who decides the value of 4 people in Riverdale vs 4 people in Beverly Hills? Just going by “frequency of shootings” is a form of bias.
"Unbiased" is like absolute zero. You never actually get there, but don't think there is no distinction between putting in an effort to make something decently cold, and hurling it into the sun.
Imagine, you had some tool affording you complete information about everything happening on earth, aided by some AI that only tells the truth about anything when asked.
You would still have to pose the right questions or risk dying in ignorance.
People's complaints here about bias is really about being manipulated successfully by parts of society working actively in their own, markedly different, interests.
Those won't just go away. Aligning interests doesn't happen by providing facts.
I realise now that my messaging with this project is off.
What would more accurately explain the purpose of this project is to reduce the effects of sensationalism and indoctrination in the news.
In other words, provide a platform for readers to make up their own opinions on news stories by filtering out the excessive positive/negative language.
Even if NewsNotFound articles are only 90% neutral, that's 90% BETTER than reading the same story from a sensationalist source.
If I understand correctly, you want to help people get an objective baseline of information about the world, they then can compare their individual bias against. Only thing is, I don't think most people care about that.
Humans provide their sentiment about developments to further their individual interests. Other people want to be informed about those motivations. Taking that away makes news subjectively meaningless for most.
There is a minority of truth-seekers who essentially individually engage in OSINT activity (if only by consuming more than one source).
An AI "doing the thinking for them" isn't the right way though (trust is an issue, remember). You want to teach people, how to be an intelligence analyst and give them the tools to avoid the drudgery.
Isn’t the aspiration to be unbiased worthy already?
The goal or call it “aspiration” to serve unbiased news is admirable to me, even while my rational mind knows the aspiration is an unattainable ideal. I will give them a try. Then I can judge their sincerity and performance relative to their aspiration.
It sounds to me like you want to rely on quantifiable science in all things. Yes science is founded upon math which is founded upon logic. Both math and logic are axiomatic systems, and axioms are assumed or declared. They are not independent of a person's mind either. You cannot escape it. David Hume sorted it out for us long ago. Rational certainty is a pipe dream. But that need not prevent us from navigating this world and enjoying life.
Back to the topic: we seem to be talking about a human being choosing between competing news sources. As that consumer, you get to define "unbiased" for yourself. Wise people have given it a lot of thought, so you might consider consulting what they said.
And you can draw the line at something being measurable to matter to you. That's your choice. I have found the things that matter most are fictions in my brain: love comes to mind.
It is also not necessary to measure to detect a distance. I love my dog, and I love my child. I cannot measure love, yet I detect a significant distance between my love for my dog and my love for my child.
Like love, "bias" has meaning even if it cannot be measured. Like love, one can detect distances between biases, even if they cannot be measured. Science (measurement), math (quantity), and logic (relations of ideas) are just as ephemeral, because they are founded upon unprovable axioms and norms.
Objective observation is really hard, but amenable to systems and rules that help get closer to this ideal. I'm convinced it's possible to do far better than existing news organisations do just through approaching it with an engineering mindset.
I do believe in science, I think perhaps you have a very biased, or at least limited, view of what “believing in science” means.
There is no way to judge whether or not you are close to some objective view of “unbiased news”. So it is not a worthwhile pursuit.
As others have noted, what is “newsworthy” is an entirely subjective prioritization of events and information. Are four deaths in Riverside news? Are four deaths in Palm Springs? Are both? Are neither? It depends on your subjective priorities, as a consumer, and no single news organization is ever going to achieve universal, subjective, but somehow unbiased news prioritization.
Labeling anyone person or group’s prioritization of information as “truth” is a dubious endeavor. Dubious to the point of being laughable.
I wonder if this is true in reality. Isn’t it possible and maybe even likely that a non-neutral party is providing the most accurate version of the story? For example, a person or reporter living in a war zone, experiencing the actual war is likely biased in some fashion, but their recollections of what is actually happening is quite accurate compared to a “neutral” journalist sitting in their comfortable office in New York City, referring to various anonymous sources and reports from multiple perspectives.
Also, filtering out the positive and negative language might hide the bias of the article, making it seem more authoritative and believable. Many times when I read an article, and I see the journalist making conclusions with bombastic and unbalanced language, it is helpful information toward allowing me to detect bullshit and avoid that source.
3. Information that doesn't affect me but I still care to know
4. Information that won't affect me
5. Information that won't affect me and I don't care to know
6. Information I don't care to know even if it affects me
Surely there must be a way to monetize information delivery. I would pay for an editorial-less stream of information that is relevant to me, and that isn't that I still want to know.
I used to think that unbiased news is an intractable problem purely based on information theory: to have an unbiased picture of the world, you need to have all the facts, which is physically impossible; and any layer that simplifies those facts (be it a journalist or an LLM) will add bias (either by journalist, journalist’s employer, LLM’s training data or operator preferences, and so on). In short, when your selection of facts to report is already biased, there’s not much you can do. In even shorter, map is not the territory (and all you have is a map).
However, I realized that’s not really true.
I mean, of course you cannot have unbiased news, and reevaluating the definition of “news” is not going to help you.
It’s just that there is actually a very good mechanism for combatting this—and it’s called “be aware of your bias and upfront about it”. There is any other way around it.
To add: where I wrote “be aware of your bias and upfront about it” I of course meant journalists and news organizations.
Corollary: since this awareness of the bias requires self-honesty and introspection, it’s drastically more difficult with an LLM because you also need to know all about your LLM developer and operator’s accidental and intentional biases.
Technically speaking, if every day you take an uniformly random selection of news articles, on average what you read will not be biased. Or, to be precise, it will follow the same biases as the population of news articles you sample from.
It's almost impossible to arrive at a pure random sampling like that. As an easy example, if you limit yourself to English news articles, you're going to have a pretty large bias towards "the West", which right now probably includes a strong bias against China or Russian's viewpoints on things (not to say that including those viewpoints would be more truthful necessarily, but certainly less biased).
The difference is that facts are harder to consume than news articles, as the latter is essentially packaged and distributed collections of the former.
Furthermore, the intent of parent's bias averaging, as I understand it, is...
1) There are a finite number of major biases axes in mass media (e.g. political parties, wealthy-poor, conservative-liberal, etc.), ergo averaging these out is a helpful reduction of major biases
2) It assumes the underlying material has a bias, and seeks to counteract it. If you build a fact-based method, you leave yourself open to treating biased opinions as facts, if they're sufficiently masked
Indeed, the first challenge to unbiased approach would be the impossible in practical terms task of separating facts from fiction.
I don’t know if it is clear but I really meant it that unbiased news is not a possibility due to fundamental limitations of information theory and finiteness of individual information processing capabilities; self-awareness and bias transparency is the only way.
> helpful reduction of major biases
Only if those major biases are opposite between media.
Imagine biases as vectors pointing in arbitrary directions. Best case is when bias vector in one cluster is compensated by exactly opposite bias vector in another cluster, but how often do you expect that to happen? A more typical scenario is a vector pointing in merely a different, rather than opposite, direction. A worst-case scenario, and also common (because bias manifests itself in selection if facts to report, and most sources report on the same events), is when all bias vectors point in a similar direction.
In your original (upthread) argument that "unbiased news is not a possibility", are you stating that as a fundamental consequence of the news' lossy compression of {all facts} into a smaller product... or something else? It's unclear.
As for averaging out, yes, the vector model was what I was using internally.
Disclosure: I'm US-based, so our politics are traditionally bipolar as a result of our election system (in contrast to Europe et al.).
I would be surprised if vectors don't cancel often though, because think what we're really talking about in a capitalist information space: popular perspectives that are profitably marketable.
What is the easiest way to find an audience? Do what your competitors are not.
So on the whole, high revenue and production cost media (tv, web) in a capitalist information space will converge into bipolar pairs on any given issue, in order to maximize market share.
In lower cost media or operations, it will more likely be a random scattering of vectors.
> Disclosure: I'm US-based, so our politics are traditionally bipolar as a result of our election system (in contrast to Europe et al.).
There's a whole lot of bias that's "baked in" with US politics / media. There's broad consensus on neoliberal capitalism, personal freedoms vs collectivism, common ground on opposing states like Russia and China.
On a few wedge issues you will average out to a common ground, but ultimately you're not eliminating bias, you're just eliminating a relatively small bias between left-wing and right-wing US and replacing it with an "overall US" bias.
From mass market sources, true. Probably because they're aiming towards the common American, who generally has similar feelings. Ergo, they're that + whichever politically-polarized biases.
As for whether averaging that out is useful...?
I'd be happy to have news from a "overall US" bias, without the right/left political spin.
1. Identify all clusters of power and opinion, eg. Republicans, Democrats, Chinese Communist Party, UK Labour Party, Green Party, etc.
2. Associate each cluster with news organizations that tightly align with their policies.
3. Evaluate stories - headlines in only one cluster (under reported elsewhere) represent a view. Headlines that have different takes, likewise, give you a spectrum of thought on an issue or topic.
4. Collect fine points and build a report. Bonus for providing contrasting opinions and associating them with the party/power responsible for that thinking.
I want this tool. It'd be okay if it wasn't perfect. Just getting 80% of the way there would be incredibly valuable. I'd pay for it.
1. Will your cluster identification be unbiased? How do you avoid providing more fine-grained taxonomy for clusters that hit close to home for you (or this tool’s operator) and shoving the rest into as few buckets as possible (because you don’t care)? 2. What about biases shared by all news organizations, regardless of cluster?
When I started working on Zeitgaist [1] I immediately recognized that biased information is the main problem. I'm currently thinking about attaching additional information to the sources I present like ground [2] is doing with political spectrums. I really like your idea for the news algorithm. If you or someone elsw wants to build something like that on top of Zeitgaist with me, don't hesitate to contact me.
I don't see why you can't. Throw religion in while you're at it. Even interest groups around other classes: disability, socioeconomic, etc.
You can absolutely say a newspaper (or individual columnist or editor) is pro-LGBT, pro-Israel, pro-Palestine, pro-Mormon, pro-Amish, pro-Atheist, etc. That shouldn't be controversial. Wikipedia can and does do this in a relatively neutral way.
If your classification is meant to disparage a group, then I'd see the trouble. Don't do that.
I want an up to date, well rounded view of how everyone sees the world. A system capable of getting me to see geopolitical events in other perspectives is useful, even if I ultimately choose to uphold my existing beliefs.
FWIW, I subscribe to all sorts of diverse political view subreddits and Twitter handles. Unfortunately, it's still hardly comprehensive is a lot of work to analyze and distill.
What we're really talking about here is boting any deviation from a tabula rasa appraisal of reality, then clustering that into an affiliation/quality, right?
I.e. Christians (as an averaged whole) can look at the same situation as a blank slate and have a statistically significant and consistently different perspective
As can scientists. As can maybe microbiologists and industrial chemists.
The point of the clustering isn't that the tags are pejorative, but that they're predictive metadata on any source's bias.
And often times that's currently unexposed. I see a post by echelon, but only by the content of that single post (and maybe some prior HN contact) can I reconstruct his typical perspectives.
Because the project is small right now (and makes no money), I've included a very limited list of sources for each category, however at scale we would have alot of options for getting past this.
For example, one way in theory is to scrape all known publicly available news sites and create an article for every single detected story, but this would be very expensive.
As someone with experience in the media, what steps would you take to find these under-reported stories? Maybe the steps can be replicated with AI to increase coverage?
I think the The idea of this is interesting but I agree with you, Using titles to correlate results that are fact based isn't a greatest way to find unbiased information. You would still need some kind of variable that was not so well known. echo chambers are created by repeating information in similar ways. Its been my experience that echo chambers usually are the least reliable.
Your comment reminded me of this quote from Howard Zinn (who had his own biases, of course):
But there is no such thing as a pure fact, innocent of interpretation. Behind every fact presented to the world—by a teacher, a writer, anyone—is a judgment. The judgment that has been made is that this fact is important, and that other facts, omitted, are not important.
What are some of your thoughts on the subject?
How can somebody achieve unbiased news without editorial supervision? Would a quorum solve the issue with editorial biases creeping into news?
"By replacing the human middle-man with our AI-centric article creation process, we eliminate the potential for bias and ensure that all coverage remains impartial."
AI LLMs inherit biases from their training data and the constraints the model builders added. There's no such thing as unbiased reporting; the usual way that this is attempted is to claim that the "unbiased" position is halfway between the positions of the two dominant political parties in some country, which might look like intense bias to someone in another country.
This ideology of “replace corrupt institutions with tech” is at the center of SV ideology going back to the 70s. It’s as flawed then as it is now, but it occasionally has wins.
There’s a great documentary on the long history of this idea in tech, both wins and losses, called “all watched over by machines of loving grace”. Brilliant documentary.
Most obviously the tech industry has disempowered the telco business. They are now just "dumb pipes", whereas once they wanted to be all singing all dancing content providers. Wikipedia replaced Britannica. Linux has largely replaced Microsoft. iTunes Store and Spotify took away a lot of the power that RIAA had over artists.
If you want to focus purely on governmental institutions then encryption put intel agencies / corrupt police on the back foot and the digitization of processes especially payments has killed off a lot of low level corruption.
I came here to comment on that paragraph. The training data, the model parameters, the prompt, the source articles and headlines—all of these add to the bias in a system like this.
A news site saying "we eliminate the potential for bias" is like an airline saying "we eliminate the potential for accidents". It's a massive red flag and should warn you to stay far away. Bias can be controlled, but never eliminated, and anyone who thinks they've gotten rid of it is either lying or hopelessly naive.
The fact that major LLMs will happily criticize certain politicians but claim they’re unable to comment on others given equally available training data is a fairly good window into what kind of biases exist within them.
A lot of these discoveries were being made when OpenAI and ChatGPT were pretty new to the general public. Nobody knew what it did, so these kinds of questions were no different than any questions being asked, basically.
That so deep a bias was discovered so quickly was likely incredibly surprising to some, and incredibly unsurprising to others, but they were asked for the same reason as others - to find out what it does and whether it could be seen as reliable for a source of information
I suppose. I remember my first encounters with the Open AI playground were along the lines of "write a short story about X". Amusing at first but the novelty quickly wears off.
If you're using a model that's behind a moderation service in their API, you're not actually assessing the biases of the model but rather of the corporation that's trying to remain profitable.
It's unclear to me if that is an issue in the bias of the training data or an issue in the fine tuning to control the output.
Is it "I can't say that about X because I am told not to say things like that about X"?
or is it "I can't say that about X because I am told not to encourage behaviour Y and my training data indicates that comments like that about X lead to behaviour Y"?
I mean yes, it's biased compared to the mythical unattainable standard of being completely unbiased. But that's not the bar. The bar is mainstream news outlets, which are drastically easier to outperform even with a half-assed attempt.
"News organizations are terrible, so we take all of their work actually investigating, interviewing, and reporting about events, and run it through a GPT-powered article spinner without having to leave the house or do any original reporting"
Doesn't really sound like a news company to me.
Also, while I'm on the topic, "unbiased" news is an anti-pattern. A big part of the value of reporters is synthesizing and contextualizing events. Wikipedia current events[1] is basically unbiased news, but it has a tiny audience (and is still dependent on linking out to actual news organizations).
"Unbiased" news is impossible; the reporter has biases, the editor has biases, the publisher has biases; which they may or may not be aware of. The biases they don't know about are the most insidious.
So I'm most suspicious of sources that brag about their objectivity. I prefer sources that wear their bias on their sleeve (provided they're actually news sources, and not overt propaganda or gutter tabloids).
I'd much rather have one that tried to be objective, than just one that embraces their bias. The later one will always be terrible at reporting. The former at least has a chance of being good.
Seriously, the bar is not exactly high. Start with telling me where and when something happen, make a time line if it's a series of events. Tell me who is involved. Don't dress it in flowery language, make it as simple as possible, make it a list if you can. Do proper follow ups, not just event based reporting (i.e. tell me why the airplane crashed when the report is out, don't waste too much time with "we don't know anything, but here is a picture of smoke"). And of course list your sources (i.e. don't report on science paper without even linking it).
So far Wikipedia is the only one that gets somewhat close to this ideal. Almost everything else is just attention seeking clickbait.
> So far Wikipedia is the only one that gets somewhat close to this ideal.
Wikipedia now has this expectation that any claim that's at all surprising or interesting should be cited to "reliable sources". I don't entirely agree with WP's notion of what sources are to be considered reliable, but at least the policy is out there, and can be debated.
This citation policy wasn't always there, and it has made a huge difference. For example, anyone can come along and remove surprising statements from articles if they aren't supported by a reliable source. That in turn means that if you find a statement surprising, you can go and check the source.
I used to think this but increasingly I think it's a fool's errand.
Wikipedia might have been like that in the past but it's long been corrupted. E.g. see the Philip Cross affair.
For a while on the Ukraine war I tried to find unbiased commentary and news but I think it was basically impossible.
The best I could find after looking all over was stitching together stories from sources who were biased to one side but still at least somewhat committed to the truth.
FWIW I was unaware of the Philip Cross thing, but after digging into it it appears that Wikipedia's arbitration committee imposed an indefinite ban on him from editing any page regarding modern British politics [1] and that after he was found repeatedly violating it, that was upgraded to an indefinite block [2]. Not exculpatory, but at the very least evidence that Wikipedia is capable of taking issues of editor bias seriously and not sweeping it under the rug.
> For a while on the Ukraine war I tried to find unbiased commentary
There's no hope of finding unbiased commentary on a war that's happening NOW. Even the history of wars that happened 1,000 years ago are hopelessly muddied by contemporary political agendas.
As far as Wikipedia is concerned, history is one of the areas that's least trustworthy; lots of history articles have teams of gatekeepers. It can help to read the talk pages (and the archives of the talk pages) to find out what's been disputed.
The fact that you can do that, and also inspect the entire edit history, really does make Wikipedia head and shoulders better than any other information resource, provided you read it with due scepticism.
As an experiment, let's just think of the news industry as specialising in those two business lines. ^_^
Bias cannot be eliminated from the news business -- bias is the business. Even the sincerity of declared bias is no guarantee of good faith/ethics.
Consider the challenges that Wikipedia faces trying to represent knowledge about history and events in an objective manner.
"News" means different scope at different levels of conception.
* NL0 = personal, individual; at this level, "news" is the for-profit business of feeding (and shaping) the individual mass of feelings and stubborn opinions; the only objective things at NL0 are Maslow's hierarchy and *the local weather*.
* NL1 = regional power structure; at this level, "news" becomes part of regional economics, social policy, and broad weather patterns; political theatre begins obvious pandering at this level.
* NL2 = national power structure; at this level, national economic and social policy, regional economic tensions, perspective of international power structures and spheres of influence; political pandering is intense.
* NL3 = international power structures; at this level, various nations compete and/or cooperate economically and militarily; there are spheres of geopolitical influence.
News business will ally itself with a power structure. This is where democracy, for all its challenges, is still the best hope of any free (or free-ish) society.[1]
News "for the people" really depends on how militant the people are prepared to be.
So, for the most part, power structures write the news. State-funded news is only as reliable as popular confidence in the state providing the funding.[2]
Yes, but it's perfectly valid to attempt to minimise biases. Simple things such as fact-checking and not using loaded language can go a long way to becoming more neutral.
Simple things that you mention are great. But probably the nastiest kind of bias is the choice of what news to deliver in the first place; it's hard to detect bias in a news source from what it isn't reporting.
The bias in news reporting is an aspect of the sentiment the respective individuals have towards that news.
Socially, that bias is a more important part than the news' factual content itself, making comments here about how to "eliminate" it appear as a weird display of cluelessness.
People rightfully wish for a harmonious society. Eliminating other people's utterances of discontent is an approach usually employed in dictatorships.
> an aspect of the sentiment the respective individuals
I disagree. I'm a regular reader of Guardian Online; that organ has multiple institutional biases. Stories get spiked if they don't align with that bias. Reporters are forced out if they fail to toe the line.
It's not just The Guardian; all commercial news sources share this property.
I meant "respective individuals" in the sense, it's lastly humans being responsible for the written piece of news.
I find it very curious you would single out "The Guardian" of all publications. That organ is maybe the most independent one you will find today. It is owned by "The Scott Trust Limited", successor of the similarly named trust created to shield it from commercial interests.
Calling it a "commercial news source", as if it was the same as Murdoch's ilk, is an absurd take, even derogatory.
Do you say that, because it is "centre-left"? What right-leaning source would you call "unbiased"?
I singled out The Guardian because it's the one I read daily, to learn what happened yesterday.
The Guardian (and The Scott Trust) is institutionally biased. It backed the Balfour Declaration, and to this day it remains pro-zionist. It's also pro-war, and backed the invasion of Iraq. It's got a lot worse since the Assange dremels-and-angle-grinders affair; it has become the primary mouthpiece of the UK security services.
It has the reputation of being a leftie paper; but that's an illusion. It holds up the left end of the Overton Window, and reliably backs the extreme right wing of the Labour Party. Its attacks on Jeremy Corbyn were consistent and cringe-making; even the token leftists in the newsroom were forced to join the pile-in.
I call it a commercial news source, because that's exactly what it is; it's a business.
I don't call any source "unbiased"; as I said up-thread, I don't think unbiased reporting is possible.
The two sentences in the article that tells me this will be a failure:
“ Once we have a list of related headlines, we then use GPT-3 to generate a list of facts, not opinions, that are present within each of those headline’s respective articles.”
“ We then run the article through a “bias checker,” which uses sentiment analysis to rate the sentiment polarity of the generated article.”
As expected, it has trouble with time and sequence. See
"SpaceX Starship rocket system explodes during test flight".[1]
"On Tuesday, SpaceX conducted an uncrewed sub-orbital test of its Starship rocket system in Texas. The spacecraft, which is designed to carry up to 100 astronauts to Mars, exploded within minutes of takeoff."
"100 astronauts to Mars?"
The launch was originally scheduled for Tuesday, April 18, but was delayed until Thursday, April 20. The story appears on NewsNotFound on Friday, April 21, as a current event.
There needs to be a step in this process which constructs timelines.
I've read all comments and I realise that I'm presenting this project completely wrong.
It's theoretically impossible to be 100% unbiased, and I'm currently misleading people by stating this across the site.
However, it's not impossible, and quite easy, to be 'the most unbiased' news source.
I believe the project can and will have a net-positive impact, by reducing the effects of sensationalism in the news.
Even if NewsNotFound articles were only 90% neutral, that's 90% BETTER than reading the same story from a sensationalist source.
This project has the potential to provide a platform for readers to digest the news in a 'less biased' way than anywhere else on the internet.
Whether people want this, or prefer to read from biased sources because it's more emotive and relatable, is another issue and I think will stay a matter of preference.
I will update all socials, whitepaper, etc. to reflect this and I apologise for my ignorance initially.
There are a few other fundamental issues that people have pointed out, which I will try to fix over time.
> However, it's not impossible, and quite easy, to be 'the most unbiased' news source.
It's great that you are committed to providing less biased news.
But AI is not less biased, and the first step is to realize that you are not magically less biased than all the other humans who produce news. Unless you see your own bias, you can't manage for it. In fact, I think a good number of journlists do that quite well.
So do you, implicitly: Where will NewsNotFound get the 'facts' that it publishes, other than the journalists who go out, gather it, and report it? You can't witness it and report it yourself, beyond a tiny fraction of events.
Well there's been quite a backlash already, but I get the impression this is a well-motivated but perhaps a little naive attempt to do exactly what it says on the tin.
Some of the criticisms are harsh but fair, yet I think that anybody who takes the time to read the "white paper" and acknowledge the admirable dedication to building in the open will see there is genuine potential and passion in this occasionally mad project.
I love the energy and hope the (young) creators won't be too downhearted by the equal passion in some of the negative responses; a good indicator that the valid points about inherent AI and other bias have been listened to will be an updating of some of the language and soundbites used to sell this interesting project.
On a more personal response, I thought the articles I tried were really pretty good in terms of being "free from bias", with the obvious negative corollary that they were rather dry.
Bias is ok, it comes with passion often, it becomes a problem when it's not seen or overwhelms and controls.
The articles were also quite long, which may not be appealing drained of emotion and creativity; I think the At A Glance summaries will prove to be the most valuable. The pictures were mad, I liked them though.
Cool project overall, with the obvious issues noted.
I normally try and be positive / open to flawed ideas or initiatives but in this particular case I think describing it as "a little naive" massively understates it. Well motivated it might be, the naivety is gargantuan. This isn't a nice idea with some flaws, this is a fundamentally misguided idea at its core. Utterly irredeemable.
I wish the authors luck in that I hope they can step away & move on to something more worthwhile without being hurt or becoming defensive about criticism. Given the level of backlash in the comments here, its understandable that may be tough to do, but I really wouldn't want to encourage them to continue.
When enjoying good and insightful articles, I always assume that biases are deeply ingrained before an article was written. Author's educational background and affiliation plays a large role in how they write. Some of the biases are easy to spot and we can just laugh it out, but some of these are also unconscious bias. I would more likely to have an "AI" that identifies biases by analyzing the history of the authors, agencies, and related historical articles. It seems impossible to me as this is very aggressive on people who are working in the industry and a disrespect of their personhood.
On the other hand, knowing what's happening in the world in a few sentenses is another thing, having GPT to check the bias and do some cross-ref checks is really sweet, after all I don't really care about most of the news.
this is a PR piece from a local news site. There is no critical thought as to why this is in the news.
Not only is this not news, but it contains about 4 facts. The rest is vague useless information.
If you want good quality news, you need to pay for it, from multiple sources.
Contrary to popular opinion, bias is actually pretty easy to spot, its just that you need more than one source to find it. This approach just isn't going to help, it just strips away context from any bias and presents them as stand alone sentences.
But it needs more than that, GPT isn't very good a summarisation. None of these summaries are useful, and there is nothing stopping someone filling it with bias.
My favorite part is that step five allegedly ensures that the generated article is completely unbiased, and then the next step is immediately to run it through a bias checker. How do you write those sentences back to back and not realize something has gone awry?
Sounds like a great idea, but much of the bias in news is by way of lies of omission. For a recent example, last month BBC ran the headline "British woman, daughters, killed on road north of Foobar city". From the headline, one might never come to the conclusion that their car was shot by a passing vehicle, which then turned around and returned to shoot some more even after their vehicle crashed. And even the accompanying article did not make that clear until very deep in the article - and even then omitted much context.
Similarly, a few days later security forces went to arrest the perpetrators of the attack, and were themselves attacked with Molotov cocktails and large stones thrown from a high floor. They returned fire, and one of the attackers, a 15 year old boy, was killed. The BBC headline read "SpecificCountry soldiers murder 15 year old Foobar boy". No mention was made of the attack on the security forces, nor of the attack on the car to which the security forces were responding. At least, not in the first few paragraphs that I read, which mostly concerned themselves with other incidents in which Foobar citizens were hurt or killed by SpecificCountry forces, with absolutely no context.
Nothing published was an outright lie. But the omission paints a bias by itself. How could this AI-powered "unbiased news company" form an unbiased version of events, when the relevant information is not even provided by the sources?
I had family that worked at a news paper back before the boom of the internet, and something that most people nowadays don’t seem aware of is that even then, most news papers relied heavily on something called the Associated Press (AP). If you ever read a news paper and saw the symbols “(AP)”, this meant that the story was picked up from “the wire”. That wire was reference to telegraph technology from the 1800s, and it carried a plethora of stories submitted from associate news papers around the US. Given the distributed nature of “biases”, it is considered collectively to be “in the middle”.
Going deeper in terms of analysis, in the US, the PBS news hour is the highest quality.
Internationally, I read Reuters which seems similar to how the AP works, but I don’t have anything to compare it to.
Well, it's not news, the articles are insufferable.
But it's an admirable experiment. Impressive that Josh Waller (that's the name on the github repo) went all the way with this.
My initial impression was that it is some kind of satire but he seems to be playing it straight. If that's the case then I just don't get the preoccupation with "unbiased".
Journalism always has a bias at some level. Whether it becomes a problem that needs to be addressed is a matter of degree and intent. I would even say that streams of "just facts" without context, without some point of view are uninteresting to a reader unless they're performing certain kinds of tasks which benefit from such a stream-- but that is very different from "news".
In addition to all the other comments pointing out how stupid this is, and how stupid your claims of non-bias are:
Why pick such a stupid name? The Twitter handle is already taken by a parody account titled "Real FakeNews", that didn't tip you off as to how this name would be received?
> The headline should use completely neutral and unbiased language. Aim to be factual, not opinionated.
This sounds incredibly dull and lifeless. If you’re writing an encyclopedia, it’s fine. If you’re looking to be acquihired by Bloomberg Wire, it’s fine.
But, modern chatbot aren’t capable of adhering to the restriction “factual”, and will hallucinate information just as readily as a human witness might during an interview. A few paragraphs of instruction don’t change the basic nature of multi-layer network synthesis. For that, you need a rigorous training and a network of peers ready to check you when you are led astray by your basic nature — and that requires having multiple viewpoints to
mitigate groupthink.
I mean no disrespect, but I hope this project fails.
"Neutral" language isn't even necessarily unbiased. "Leader of Nazi Germany Adolf Hitler committed horrible and evil actions" isn't "neutral" but it is a statement of clear objective fact.
It's the same reason why passive voice writing such as "police-involved shooting" are problematic and biased.
Personally I think the most dangerous kind of bias is financial biases.
Luckily there are multiple news orgs not beholden such things like Al-Jazeera, although its funded by the qatari government its practically independent with many awards and reporting representing the middle east, that's why they were able to stand against american lies in Iraq(to the point the U.S. requited Qatar to limit them), and truly cover the occupation in Palestine.
It's not hard to discern bias within some article. Having more appropriate headlines is a good step, though.
But I think the bigger issue is more about the coverage of stories. Outlets are certainly going to cater their audience. Example: maybe from sources you see more stories on white-on-black shootings, and in another source you see more black-on-white shootings. So which one is more "newsworthy"? That's where a lot of bias comes into play.
> Step 1: For each category, we begin by scraping the latest headlines from publicly available news sites.
That said, I'd like to know the categories and what sites are scraped, because there may be bias in the selection of such.
> Step 2: Using sentence embedding, we identify any headlines that are related and choose these headlines as the subject of the article. The idea here is that if a story has multiple related headlines across different sources, it is more likely to be of greater importance than a story with none or only one related headline. This helps us determine which story to focus on and write an article about.
Step 2 seems like a process to directly bias the inputs based on the popularity of each outlet. Or rather two processes, one hidden, because the outlets aren't selecting themselves. Why wouldn't this just be a wire service and press release detector?
It’s kind cool in that it:
- removes/neutralizes words that tends to flavor articles toward any particular position and focuses mainly on factual non-emotional type language.
- sends blurbs out in a daily email (arguably editorial) - but generally tends to be well rounded important stories to keep you in the know about what’s going on
It’s pretty cool, because unlike most new sources you don’t get a feeling anyone’s trying to railroad you down a particular way of thinking, but you feel like you’re just being presented with (more or less) statements about what is going on in the world.
Honestly, I think "less-biased" would be more appropriate than "unbiased" and lead to less criticism.
Otherwise cool project, I like the style of the generated illustrations.
I'd rather a known and well-established bias than the algorithmic illusion of no bias. If I know the bias of my news provider, I can read them critically fairly easily. If I have to guess at the bias, that's much more difficult.
I think a more useful approach would be to aggregate stories from different sources together and summarize the agreed and contested information and viewpoints. That would be a useful tool for evaluating news and identifying propaganda narratives.
It seems to me that their view of bias is limited to a per-article approach. This could make it succeptible to broader narrative bias across stories - if, for example, the news sources are selectively only reporting on muggings when perpetrated by one race/nationality/immigration status/gender over another then it'll still give a false impression of attacks by those people, as it'll be completely blind to the lesser-reported cases.
So, if this project were to be successful, it would replace news media as the outlet where people would look for news, thereby, reducing audience for the news media it's using as the source for it's information, which will provide less information due to reduced resources, and even more interestingly, get ever more sensationalist in trying to attract readers thereby increasing the bias of other media the AI depends on for information.
Is it unbiased in the sense of the word, or in the sense that the US-far right uses the word where everything that doesn't support the far right narratives has a "liberal bias"?
(meaning media that doesn't fact check their lies and doesn't tell facts they dislike)
If you tell a chatbot trained on the internet to be unbiased how do you know which definition it will apply?
After the skip to contents link, there are five unlabeled links, and after that is the news not found logo link. After the contact link, there is an unlabeled button, which is followed by the heading "NewsNotFound: An AI-Powered, Unbiased News Website". If these can be fixed, I think this could be a nice accessible site.
I'm politically biassed but this company does a reasonable job of being un-biassed and even-handed. It's not the Economist but if you just a want a few interesting stories per day you could do worse: https://www.rocanews.com/
What if "Bias is inherent and unavoidable in all Human Thought & Experience" is the correct answer?
How can any particular Individual be "unbiased"? It's impossible. Yet only an individual is "alive" and "living" the way it is legally/socially defined.
It’s up to the individual to diversify and triangulate their news sources. Open source news and open source software are two different things, so keep that in mind when parsing this. It’s up to everyone to call out poor journalism when they see it. That includes omissions and conflations.
Not terribly good, yet: searched for “Canada”, first story was about new ant fossils, second about storms sweeping US and Canada, the next several were about Dominion and Fox.
So two “meh” and many “wtf”. Refinements remain to be made.
>The only news you should be paying attention to is local news
It's time to completely obliterate this meme. Local news are not relevant. They never were, they never will be. Let's see what my local newspaper has to say for today:
- weather today (probably less acurate than other sources)
- local grandma is about to turn 103
- car crash in midtown (nobody was injured)
- motorcyclist crashed (minor injuries)
- the results of a soccer match nobody cares about
- some section of the park will be renewed
- mishmash of national news and international news (copy pasted)
Even if someone you care about has an accident, you'll find out before you check this. The only thing would be a site that keeps track of the city's history or something like that. That and ads for local businesses... and obituaries.
I really like the idea! I don't even mind that it hallucinates a little bit since I remember probably 10% of what I read and I'll read multiple sources for things that matter to me anyway. As for everyone commenting on bias: we all know there's no such thing as 0 bias.
Imagine you could see into the minds of news reporters - their intentions, thought processes, and emotions when writing articles.
This would allow us to hold reporters accountable for bias, misinformation, and sensationalism.
Now imagine we could also dictate and control these thought processes and intentions, so as to ensure they aren't aligned with any particular viewpoint or political agenda.
Unfortunately, this isn't possible in humans (yet).
NewsNotFound proposes a solution to this problem, by removing the human middle-man between you and the news.
With open-sourced code, anyone can now see exactly what goes on behind the scenes of article creation.
This is the key to a trusted news company - complete transparency and the opportunity for anyone, anywhere to contribute.
This idea punches down. Journalists work so hard to complete the work they do. They do not need this.
And as noted elsewhere, AI is not inherently unbiased. In fact, one of the big problems with AI is that it introduces systemic bias, and understanding the role of fairness is very important here. https://www.vox.com/future-perfect/22916602/ai-bias-fairness...
I would also raise the point that journalism is actually moving past this idea of objectivity at all costs, where writers intentionally wear their point of view on their sleeve. It is no longer a negative to be purely unbiased. In fact, there are real concerns that objectivity leads to important questions not being asked or presented. (You might have heard the term “the view from nowhere.” That’s what I’m getting at.)
To be clear, the work of journalists should not go unchallenged, but this feels like something you should spend time actually talking to some experts on journalism before you actually build it. The mindset you’re presenting here does not suggest you have.
This doesn’t replace journalists, it’s just an AI driven link aggregator.
That said as others have mentioned, LLVM’s are inherently biased through their training and this site only picks from moderate and left wing sites, bringing into question whether it could ever be unbiased.
Its positioning is designed to question the legitimacy and objectivity of traditional journalism while positioning itself as an alternative. That deserves more pushback than the average aggregator does.
Just trying to settle on a definition on left, moderate, etc would grind this thing to a halt. Most of what people call left is center by any view I would consider well-founded. Then there are places where what passes for center in the US would be far right.
Not trying to classify would make choosing sources that are actually representative impossible unless the creator were operating under the illusion that unbiased sources are a thing.
Honestly I agree, it's a matter of perspective and even most self described left or right-wing outlets will have journalists who occasionally write articles that swing the other way.
Further I want to add I don't think this is a bad idea, I just think the idea of collecting "unbiased news", although desirable, is unfortunately impossible. The idea and model they've developed however may be very good.
This is referring to how the open-sourced code provides the freedom for anyone to criticise or correct the way the site works through contribution.
Also, LLMs are biased, but through prompt engineering and other techniques, you can produce neutrally-written text.
I realise now that my messaging with this project is off. What would more accurately explain the purpose of this project is to reduce the effects of sensationalism and indoctrination in the news.
In other words, provide a platform for readers to make up their own opinions on news stories by filtering out the positive/negative language and putting as much focus on facts as possible.
Even if NewsNotFound articles are only 90% neutral, that's 90% BETTER than reading the same story from a sensationalist source.
> through prompt engineering and other techniques, you can produce neutrally-written text
Through thinking and editing, humans can (and often do) produce neutrally written text.
What does AI change? Also, who decides what's neutral? The AI's programmers and operators - humans. Who decides if your AI's output is neutral - you do. Why couldn't you do the same with a human writer?
> Even if NewsNotFound articles are only 90% neutral, that's 90% BETTER than reading the same story from a sensationalist source.
Nothing is 100% sensationalistic. Anyway, I can (and do) find endless non-sensationalistic news.
AI changes nothing; you are an editor (which is a good thing to be - I wish you well).
But the idea that AI is somehow more neutral than humans, or that humans aren't in control here, is very dangerous. That is my concern.
Haven't you heard? The era of expertise is over! Anyone's opinion is exactly as valuable as anyone else's - in fact, the opinion of someone who is not an expert is _more_ valuable than that of an expert!
That's often true though. One way the media biases articles is by constant reliance on people presented as neutral experts but who don't satisfy the definition used by most people. Normally an expert is defined as someone who displays unusual skill or knowledge within a domain. The media tends to define it very differently, as "someone working for a public sector institution or NGO who claims to be an expert". No actual objective test of expertise is done before presenting that person's views as the final word on a topic and if the views of that person are being challenged by other people who know a lot but aren't institutionally affiliated, then that is just ignored entirely.
Their agenda is evident in their claim that AI is unbiased.
It's hard to believe that someone who works with AI to this degree would think for a moment that it is unbiased. It's an attempt to deceive the ignorant, for whatever purpose.
From my own experience, I can tell you that the most important news stories are often the ones that go unreported/underreported by most news sites. Unfortunately, there are also many insidious methods used to shape public opinion, such as distraction etc. But it all starts with the decisions made by editors about WHAT YOU, as a reader, should SEE and CARE about. So if we truly want to achieve unbiased news, we need to start by reevaluating the very definition of what is news.