Hacker News new | past | comments | ask | show | jobs | submit login
The Rise and Contentious Fork of RSS (twobithistory.org)
253 points by ttepasse on Sept 17, 2018 | hide | past | favorite | 186 comments



I still use RSS and it's, in many ways, better than the web stupidity that surrounds us.

The title is misleading.

People who captured the feedreader market, killed the competitors then dropped it did a lot of damage. Maybe deliberately. LESSON: Never trust a big company that relies on advertising.

RSS is a way to escape the nonsense that has come with those, who have no idea what the web can be, but have flooded in and drowned so much in their ignorance and lack of vision.


Having worked at a giant company, I know there are many competing motivations, teams, and personalities involved - companies are not uniform entities.

Still...

Google Reader did more to destroy my perception of Google's reputation than everything else evil they've ever done. Conspiracy to suppress wages for tech workers, James Damore, the Snowden revelations - all of those bugged me, but not like Reader.

They killed Reader, I suspect to support Plus, just as the walled gardens of the social networks turned into iron curtains, destroying the only viable alternative decentralized content distribution mechanism. And now we get to live in that world.


Other than my own list of unread bookmarks, Google Readers signal to noise ratio (as a platform) was second to none. The feeds I followed would post 95% worthwhile content. A blog post is like a dinner speech, don't want to miss it. In social media the dinner speech is also available, only sandwiched somewhere between all the small talk and pleasantries. I never found a replacement, and quite a few people I followed cut down on blogging.


https://bazqux.com/

That's the closest I've found to the cleanliness and speed of Google Reader, with additional features


Feedly still works. I wouldn't be able to follow as many blogs as I do without it.


Their pricing and paid features scare me, because it's clearly intended for professional news gatherers, which means I'm not their actual customer.


In terms of tech evilness, those aren't really comparable, they're more political.

Reader's shutdown, after monopolizing neAly RSS tech and in order to push their own flavor of the year (Plus), was a classic tech evil

The other evil that Google is perpetrating is AMP. Reader followed an "embrace and extinguish." AMP is a full on "embrace, extend, and extinguish."


Did ex-Microsoft people join the Google Plus team?


EWh. There are still RSS alternatives and at least most sites (if by no means all) still have RSS feeds.

Now, you might argue that "But Google Reader was the de facto standard and mainstream audiences didn't know to just use something else."

However, unless they were reading RSS in the background through a portal like Yahoo!, mainstream audiences weren't using RSS long before Google Reader was killed (if ever).

Explicit use of RSS through RSS readers was always primarily the province of journalists, analysts, researchers, and various other folks who wanted relatively unfiltered from-the-source on specific topics.

It also didn't help that individual blog content has tended to migrate to publishing platforms in general. I still write a lot but publish on my "personal" blog relatively seldom these days because visibility is so much higher elsewhere.


There were a lot of Google Reader users that didn't know anything about the details of RSS: they already had an account at the time because they were using Gmail, and they'd automatically see "shared" stories feeds they could subscribe from friends/contacts that were also Gmail users, and from those social feeds they'd see articles from other feeds they could subscribe directly. All without knowing anything about RSS or how to find an RSS feed on a webpage they visited.

Google Reader nailed the social experience around RSS, and there definitely was a high point where RSS feeds were used by a lot of people on Google Reader that didn't understand and didn't need to understand RSS feeds to take advantage of them. It was just that weird news app "next" to Gmail on their browser tabs.

At least that was my anecdotal observation. A lot of college friends that I tried to convince to use RSS for ages before suddenly used RSS because it was already lit up on their Gmail account and they were seeing the stuff I was sharing at the very least.


OK, we've rolled back the claim of the title using a representative phrase from the article.

All: let's focus on something other than the title now—it's a pretty interesting piece for anyone who doesn't know the history (as I didn't).


I wonder how much you learn from hacker news submissions. Thank you, dang!


> LESSON: Never trust a big company that relies on advertising.

Let me rephrase that. Never trust people more powerful than you to consider your situation.

That is basically also a core principle of the distributed building blocks of the internet. It's just that to the masses and to the power hungry these options are not attractive.


As I point out occasionally, most hard news sources have RSS feeds. The New York Times, Reuters, the BBC, CNN, and Fox News all have RSS feeds. The Hill, Roll Call, and Politico have RSS feeds. The Congressional leadership has an RSS feed. NASA and the ESA have RSS feeds. The National Weather Service has RSS feeds. New York City and the LAPD have RSS feeds.

If it matters, it's probably in an RSS feed. It's social media blithering that's left out. And ads.


Stuff that the creator/producer wants syndicated will probably have an RSS feed still. As social media platforms benefit from content lock in as opposed to syndication, the publisher (the platform) has no incentive to offer RSS.

I am not particularly bothered by the lack of RSS on social platforms.


Reddit has RSS feeds for every subreddit too. I wonder how long for, now that they're trying to pivot to being a more "proper" social network.


I dont know if that’s true. If I could consume Facebook via RSS, I might follow it a bot more. When I wanted to respond or like, I would have to switch to the app.


You just answer your own question? With RSS you'd only use app when you feel like to involve, but social media platforms generally want you to stay in the app, hence no incentives.


FB couldn't collect data on your usage from an RSS feed, which is what they use to drive their advertising platform, which you are also not participating in. So having you read outside of FBs site or app is not in their best interests.


You would think that small and upcoming social media sites would support RSS, at least to attract users.

Apparently Mastodon generates an Atom RSS feed for every user, https://news.ycombinator.com/item?id=16755553


Twitter used to have RSS feeds for every user. I still follow a few accounts using code that converts them to RSS but I probably would not have bothered if I didn't already have a Twitter API key.


Bingo.


> If I could consume Facebook via RSS

You used to be able to do that. The problem (from fb's point of view) as the sibling comment points out is that you'd never visit facebook.com unless you wanted to actually wanted to interact with the content.


you still can follow public FB pages with RSS. It needs some HTML parsing but it's doable. To make it easier i've added the feature to aktu.io and i'm following a few FB pages of websites that don't have RSS...


I agree, I don't know where this idea that RSS died is coming from. RSS seems to be doing just fine. I use Feedly to subscribe to dozens of feeds. Before that I used Google Reader. I subscribe to more feeds now than I did then.


Yep. Feedly on the desktop and Newsify on iOS are a great combination.


Interestingly, Bloomberg doesn't have RSS feeds.


They have some. I use RSS to get Matt Levine's column, for example.

Link: https://www.bloomberg.com/view/rss/contributors/matt-levine....



No, they don't have an official list of RSS feeds for all categories, like Reuters[1] and others do. There's no way to follow bloomberg.com/markets via RSS, for instance.

[1] https://www.reuters.com/tools/rss


I found this, perhaps it can be of some help https://www.bloomberg.com/view/rss

Oh, and there's also this https://www.bloomberg.com/politics/feeds/site.xml


Back in 2006 I wrote an extremely detailed history of some of the infighting that shaped the history of RSS. It was a popular article at the time. For anyone interested in this bit of history, written at a formative moment, the article is here:

http://www.smashcompany.com/technology/rss-has-been-damaged-...


This was a great read, thank you!


RSS never died. On the contrary, its usage for content aggregation & monitoring has been increasing over the years, specially in the business world and niche markets.

I'm speaking from my own experience running Feedity - https://feedity.com, a growing startup that helps with custom feeds for unstructured sources like webpages.


I've been wondering why Feedly, which I use exclusively to follow webcomics, has been hammering me with ad copy about "the news you need to stay on the cutting edge" lately.

(RSS is, like, objectively the best way to follow multiple webcomics. I don't understand why it's not more popular.)


> (RSS is, like, objectively the best way to follow multiple webcomics. I don't understand why it's not more popular.)

The core feature of RSS is that it takes content and moves it from a site the publisher controls and into one the reader controls. This is immensely valuable to readers! It's perhaps less valuable to publishers, however.

Many publishers invest significantly in adding value to and gaining value from their websites. Advertisements, links to other sites, merchandise sales, etc. Almost all of those vanish when you feed only the core content to readers through RSS.

With all that in mind, publishers have ample reason to not invest in RSS. The economics of it are not kind to them.


If the "economic" publishers all agree to be monetized and harvested by Facebook and Google, this could be a self-correcting dynamic.

The only people left using RSS will be the classic web of independent readers and writers, who are still free to donate directly to each other with only a payment processor as middleparty.


If the "economic" publishers all agree to be monetized and harvested by Facebook and Google they would be mice sitting in the cat's paws.


Indeed!

At the same time, it's worth considering that those publishers are often much worse off when they escape the cat's malicious paws. More than one newspaper has learned this to their considerable sorrow.


I use RSS for webcomics as well as visiting the actual websites, but a few things:

1. You miss out on the aesthetics each comic's website brings to the table

2. Sometimes you miss things like "hidden panels" or blog posts you're interested in.

3. No advertisement money going towards the artists (Not a big deal if you support them by buying merch and the comics don't need a lot of bandwidth)

I would really love to develop a reader that focused on webcomics, having both a dynamically curated section as well as all of your subscribed comics. Newspaper-like layout. Customized borders around each comic to add back some of the flair each comic's website offers. Quick links to the artist's blog/website/comic permalink. Text/image-only advertisements interleaved between comics, with 100% revenue going back to artists. Even merch advertisements which occasionally appear under an artist's comic, offering links to their storefronts.

Basically an open source Funny Papers for the web, which would offer content discovery for artists through its curated/related sections, as well as revenue from advertisements. It would update daily and could be used to follow both new and old comics serially.

I just need someone to help me build it because I'm juggling too many things to devote all of my attention to it.


I don't think 1., 2., and 3. is applicable to at least how I use my RSS reader. Also, I don't understand why that is not the norm...

The way I use my RSS reader is that I scroll through the list of posts, and if it's something I wish to check back on later, I press the shortcut to send it to instapaper, and if it's something I wish to check now, I press the shortcut to open it in a new tab, and then I go on. When I'm done, all posts have been marked as read, and then I have a few tabs open that I immediately check out, and a few links in instapaper that I check out later. I never actually read an article or a comic or a video in my reader. It's terrible interface, I find. I much rather check out the actual website. I do the exact same for instapaper; open in a new tab and archive the link.


It sounds like you are using RSS as a way to generalize your access to each provider, but only as a delivery mechanism. The viewing is still done in the browser.

With the exception of web apps / extensions, it's my understanding that RSS feeds were originally meant to be consumed by a program specifically optimized for handling feeds, instead of a bloated, slow, unsafe web browser.

In the end, your RSS flow could mostly be replaced by a set of live bookmarks in your bookmark toolbar, then you wouldn't even have to leave the browser. Plus you would get easy multi-platform synchronization of your feeds. A hell of a lot easier than running an RSS reader on a private server like a lot of us.


Well, that might be the intention, but I have never experienced an "RSS Reader" that have left me with the want to actually consume the content inside it. I much prefer opening say an article in a browser and reading it with "reader mode" if it has bad text-layout.

Also, my RSS reader is just a web app, not a native application, so when I get through my list of new posts in my feeds, I'm already in the browser.


This is how I use RSS as well. I wound up writing my own self-hosted RSS reader which is extremely simple and supports this workflow.


Any chance you've open-sourced this solution? I've been looking for something self-hosted for ages.



I can't speak for OP but I use Tiny Tiny RSS: https://tt-rss.org/


I'm up for it


Happy to start discussing it and see where it goes. What's the best way to reach you?


It's moura at oko dot ai


As a user of feedity I'd just like to take a moment to say Thanks!


Hey thanks!


I might be biased here as i've built a news aggregator/RSS reader (https://aktu.io) but imho RSS was and still is the best way to get news from sources you like. This is so obvious when you compare RSS to social platforms... I mean Facebook was great to share stuff with your friends and get updates but now it's become a huge mess where you get a little bit of everything, you're not even sure you'll ever see all the publications from sources you like/follow thanks to obscure algorithms, and to make things worse, priority is given to images and videos...


And yet here we are on Hacker News, a social platform for news stories.

I honestly have never used a feed aggregator, I used Slashdot, then Reddit, and now a mix of Reddit and HN.

Having a filter of some sort is really useful.


And I'm here because this story appeared in my RSS feed.

I 'consume' other people's thoughts and opinions via HN, after 'consuming' the story via my feed.

Different purposes.


These both serve different purposes. RSS is for sources you like and want to see everything from. Social aggregators are for getting exposed to things from sites you're not already aware of or aren't usually interested in.

I want to read every XKCD comic, so I subscribe to its RSS feed. I like to read a little tech news when I'm bored, so I go to Hacker News.


Not sure HN qualifies as a social platform, and i got there through RSS anyway :-) Used to love Slashdot, now spending more time on HN too i guess and following some reddits through RSS. I even follow some Facebook pages through RSS so i minimize the time and energy spent on FB.


Almost all stories I comment on have at some point appeared in the HN RSS Feed, which contains basically stories that appear on the frontpage. Since I poll every 5 minute I also catch a lot of stories that only briefly are frontpage.


by the way, i'm trying with Aktu to improve the RSS experience, well at least mine :-) I've always missed a few things that i tried to add to the platform. I mostly wanted to reduce the noise in my feeds and i also wanted to have a way to be exposed to news outside my feeds. So i built Aktu in a way that same-story articles are automatically grouped together, which helps reduce the noise: instead of having 10 articles about Apple's new hardware event showing up in my feeds, i now have just one big story with references to all the articles... So at the same time, if i want, i can now easily check tens of arictles about that story, articles from my own feeds but also from sources i don't follow. There's a lot of features that i plan on adding but i'm open to suggestions and would love to hear from longtime RSS users what cool features would be great to add!


Hey, just looked at Aktu and I was very impressed with the great job you've done! How long have you been working on it? What's the tech stack you're using? Would love to hear the details.


RSS reader is used mostly as a updater of the source/site who I care about. When I read, I most of time would still go to the site but RSS allows me to quickly get a summary if I'm interested to go to the page. This is to me the best use of RSS reader, is to keep in touch with the sources. Not much a reader per se.


As long as those sources have an RSS feed (that is usable, looking at you Medium), sure. But, most sources that _most_ people (not geeks) like... don't.

The comment above where someone said "RSS isn't dead, I have a business scraping pages to make RSS feeds when they didn't have one already"... yeah, exactly.


I recently built a new OPML set of feeds from scratch, in different areas like Linux, ham radio, science, general news, and shopping/deals. Easily 95% of these sites offered RSS or Atom feeds. The main offenders were national news sites, for which there were plenty of alternatives.

Only a couple are available yet annoying, like the forum at eham.net, which offers a feed that essentially gets you unsorted forum replies mixed in with new topics. So every day's updates are full of "Re:" entries. Anyway that's been the only speed bump so far. I was surprised.

Also, since I monitor niche subreddits on Reddit, their generalized feed availability means I can automatically get amazingly wide access to a variety of niche topics (e.g. shorthand writing).

Part of my experiment was aimed at figuring out whether RSS is useful to me. It really is, though careful culling of unwanted or unneeded feeds is important. It's kind of like maintaining a plain-HTML links page. Still valuable in its simplicity, but also requiring ongoing refinements to continue to be useful.


yep RSS sure isn't the most user friendly solution, and i guess it's always been the problem #1 with RSS. But it's getting better, or at least not worse?, there's now many good web-based RSS readers that do a great job at helping people find and subscribe to sources but it's still not facebook-easy.


Podcasting is RSS. That means that 26% of Americans (I don't have worldwide stats handy) depend on RSS every month.

So, more Americans use RSS every month than use Twitter.


Tangentially, this is also a great argument in music business discussions against the finality of Spotify, Youtube and other centralized streaming platforms. People are doing just fine consuming podcast RSS feeds using players when the UX is smooth; there's no reason why that couldn't be the case for music or other verticals in general (direct artist-to-fan).


Problem with video is still bandwidth and the fact that it is more difficult to distribute.

For podcasts you need one mp3, usually quite compressed. For video you usually need more than one source, adaptable stream quality and good seeking (because most people do not watch videos end to end but only view parts of it).

This is probably not insurmountable but I guess we need better tools for creators to abstract all of this.


This is already solved by HLS - all you'd need is a .m3u8 file in the feed. Industry insistence on DRM issues is what's preventing it from taking off, but "indies" could do it.


That's quite interesting. Do you have more stuff to read about it? I never really used podcasts so I always wondered how people listen to it and assumed people use spotify or something like that.


Podcast 'subscriptions' are a perfect RSS use-case.

A 'podcatcher' (ie whatever app downloads it) is downloading the 'enclosure' url (which can point to any type of media) of the latest RSS 'item'.


> Do you have more stuff to read about it?

Yes! Check out the deck and video from Edison Research's "The Podcast Consumer 2018" at http://www.edisonresearch.com/podcast-consumer-2018/.


As a maintainer of the FeedIron plugin for Tiny Tiny RSS I see RSS as quite alive and kicking.

https://github.com/feediron/ttrss_plugin-feediron

On occasion I have needed to use a site parser like rss-bridge but honestly it's so rare that a site doesn't have rss (albeit often hidden from view). Just a hint for people searching for rss feeds the most common I find are:

/rss

/rss.xml

/feed

/feed.xml

/index.php/feed

/index?format=xml


The article downplays the significance of Atom.

Today, nearly all Web feeds are available as Atom. Hackers, learn from this. Specs matter.

https://web.archive.org/web/2004/http://diveintomark.org/arc...

Atom exists precisely because RSS is a clusterfuck that could not be salvaged.


Regarding the "The myth of RSS compatibility" article, at one point RSS was evolving and changes broke stuff. But the article overstates the case, and the RSS spec has been boringly frozen for almost two decades now.

> The article downplays the significance of Atom.

Does it? Atom Syndication Format seems like a vestigial feed format, so I'm interested in cases where it's used and RSS isn't also available.


A point not mentioned was Dave Winer's refusal to clarify if excerpts in his version of RSS feed needed to be plain text or contain HTML (either as HTML tags or entity encoded). He just kept saying "Keep it simple, people!"


I've always said that RSS is Dave Winer's gift to the world.


I type prose naturally with curly quotes (I love my Compose key!), and the fact that they aren’t as commonly mangled as straight quotes has been a nice bonus. (OK, so occasionally it’ll become mojibake, but I find that to be rarer than improper HTML entity encoding, for the last few years.)


There's no concept of an "excerpt" in RSS, but if we're talking item descriptions, the RSS 2.0 spec says that HTML is allowed and provides examples.

http://www.rssboard.org/rss-encoding-examples


Those examples came later than Dave Winer's stewardship. Those were made by the RSS Board which is its own point of weirdness in RSS' history.


You could include the whole post or just a portion in the feed, but Dave never clarified how it was to be encoded.


> Maybe RSS could have been extended somehow so that friends subscribed to the same channel could syndicate their thoughts about an article to each other.

@dang Congratulations, you just invented OStatus. :-)

OStatus is a combination of Atom+ActivityStreams+Salmon+WebFinger+WebSub for exactly this purpose. It was created in 2009 and was even partially used by Google Buzz. I could post something on identi.ca and see it pop up in my timeline on Google Buzz. But as you indicate regarding RSS, Google didn't know how to monetize it, and they soon killed Google Buzz in favor of the closed Google+, years before they killed Google Reader.

When Mastodon arrived in 2016 it used OStatus as its primary protocol until the new ActivityPub was recommended by the W3C as a replacement. OStatus is still supported by Mastodon as a secondary protocol, and part of the Fediverse still runs on pure OStatus.

Diaspora was inspired by OStatus, but removes the WebSub part in favor of going full Salmon, and does a few of the details differently.


dang?


Seems to be one of the people responsible for the article:

https://news.ycombinator.com/item?id=18003343


dang is a HN moderator. This comment is about the HN submission, not about the original article.


Since the discussion about RSS is popping up every month now, I'll just repeat myself that RSS is still doing great in terms of publisher support. See https://news.ycombinator.com/item?id=17684481


Maybe someone is paying a PR company do push the topic because they plan to release a new product around RSS. Would certainly be interesting.


Any casual HN browser has seen the scores of "Google is evil... and they killed Reader!" comments endemic to the discussions of any produt release or shuttering by the former tech darling. It was just a matter of time before some smart reader figured out how to leverage the sentiment with code and vc funding. Thats what we do.


I see a resurgence happening and a great example of that is Micro.blog -- a social network built on top of RSS, JSON Feed, and other open standards for posting and comments.

https://micro.blog/

https://help.micro.blog/2015/why-i-created-this/


Also use it every day, but there is something to be said about the issues with it, mainly that it’s invisible to most people. It’s very much a techie thing and I think the only reason it’s still around is because it’s default in a lot of site software. It’d be interesting to see what kind of engagement larger news organizations get from RSS links.


> It’s very much a techie thing and I think the only reason it’s still around is because it’s default in a lot of site software.

"WordPress powers 31% of the internet.", according to wordpress.com and Wordpress has RSS built in and ON by default.


> ... it’s invisible to most people.

that's a really general statement but I'm going to bite. Tech need not be up front and in everybody's face. I think HTTP is the same. It seems reasonable that most people don't know or care what http is or what it's doing at the beginning of urls. Or why it sometimes has an 's' on the end. And I'm even more confident saying that 'most people' don't know about how amazing TCP over IP is.

I'd like RSS/feeds of strongly typed content to be more invisible and prevalent. We're living in the HTML dark ages here.


This is a very odd comparison and I think a straw man, unless you just misunderstand. Everything you mention is internal plumbing, so of course people don’t need to care about it, by design. HTTP however does have a very “in your face” interface, which are the hypertext links themselves. People know the system exists because it is actually right there in their face. In RSS, maybe the xml is the plumbing, but there’s no clear and obvious way to access it, so much so that most people wouldn’t even know it exists. It’s hard to build and justify a system if a majority of people have no idea about it.


Just came in to say I read this in a RSS reader.


Did you read it or get a link to it? That's a subtle but important difference. I love RSS and it seems to be available every where I want it to be, even if it's a bit hidden, but it's almost always just links, maybe a snippet. The only exception in my current list is Planet GNU.


You can read it with:

https://github.com/damng/hackernews-rss-with-inlined-content

This generates an RSS feed with the contents of the articles inlined and available at https://damng.github.io/hackernews-rss-with-inlined-content/...

By-passes most soft paywalls. No Javascript. No Tracking. No 'social' media buttons. No modal dialogs. Just text you can read in a terminal based rss reader like canto.


Cool, I'll use it, at least until the hosted site gets a take-down request :)


that's ok. i'll just make a new github account or host a static file somewhere else


How do you read HN via RSS reader?



How does hn decide what articles to put on rss?


They're in the same order as the front page


If the /rss query always return the current front page order articles, then I wonder how RSS subscription services work to mark some of them read and some not.

Should probably need to know how RSS actually works.


Well formed feeds have unique id's per item, that stay the same even with title or order or timestamp changing.


I think it's actually /newest. I don't remember the RSS feed ever rearranging stories up and down the list.


> He pointed out that Twitter was basically a better RSS feed, since it could show you what people thought about an article in addition to the article itself. It allowed you to follow people and not just channels.

This is why I absolutely LOVED FriendFeed. Not only could you follow arbitrary RSS feeds and FriendFeed users, but you could group feeds into "imaginary users". This was great because you could follow anyone you wanted, even if they didn't use FriendFeed themselves.

So what happened to FriendFeed? Acquired by Facebook and then killed.


Friendica is still alive and can do this!

It's primarily microblogging-oriented, but you can subscribe to any RSS feed too. And users (including RSS feeds) can be grouped, so you could have one group "tech feeds", one group "cat accounts" etc.


This is a remarkable deep-dive into the history of RSS and the ideas and vision behind it, and how those camps diverged. But despite beginning the article with Kevin Werbach's prediction that syndication was going to become a dominant business model on the web, this never quite came to pass, and RSS took on the role of moving media between aggregators and consumers, rather than between authors and aggregators as originally intended.

With the shift in usage to the edge, RSS found itself in the company of adhoc webpages that disseminated the same content but supported scripted adtech to enable just-in-time display advertising -- advertising which, over time, became a significant source of publisher revenue. RSS was often surfaced in a different user-agent and unable to clearly accommodate scripted ads, so first-party RSS feeds were practically giving away the content for free. With time, this greatly contributed to the waning appetite of commercial publishers for RSS, despite it remaining popular in circles were the content was meant to be spread wide at no cost. Although many news organizations still offer RSS feeds, they're not really marketed prominently, and remain a loss leader to make savvy readers happy.

Since alternate business models never achieved the same uptake, efforts like Google's AMP (and its offshoots like webpackage [1]) are a practical take on the syndication concept again, where the usage of adtech is assumed from the start, and the surfacer of the content becomes decoupled from its author.

[1] https://github.com/WICG/webpackage


> syndication was going to become a dominant business model on the web, this never quite came to pass

This has happened in at least one media segment: podcasting. Ads are simply inserted in the medium proper, or the feed gets paywalled, or money is generated while producing the content (live shows).

It is entirely possible to reproduce this achievement in other media sectors, it just needs to be done wisely:

1. a catchy name ("rss" or "atom" are too geeky, "feeds" or "syndication" are too generic, "newscasting" is another thing).

2. leave clients and formats independent, so the immediate needs of users prevail over broadcasters' ones. Broadcasters are good at telling people what to think, but terrible at telling them how to use or consume anything. It also removes the petty fighting like "we won't rely on a platform built by $competitor".

3. Leave ad-management and general money-making schemes up to publishers. They can license feeds to specific clients only, they can do paywalls, micropayments, embedded ads, whatever. The client should assume nothing, simply move content from A to B on a schedule. All this can easily be implemented on HTTP codes, they are all there already.

4. The real point of feeds is exactly this: move stuff from A to B on a schedule. Metadata is almost irrelevant. Content display is a side-show. What podcasting delivers is control: it gets the content a user wants, as soon as it's available, and makes it ready to be consumed when the user needs it, in the way s/he prefers. This concept is what should move the effort, not dreams of semantic sheeps.

5. Because it optimizes pre-fetching, the format shines where on-demand fetching is time or money-expensive. Niches where it would (or does already) work well, ae things like auto-downloading TV episodes, delivering big images (something for visual artists to cut off Pixiv and friends... ?), delivering big binary updates and so on. With the rise of single-page-applications, one could even devise a format that delivers an entire website, ready to be consumed instantaneously without requiring lots of mobile data usage. Bootstrapping it in "stingy" markets (Southern EMEA etc) could help.

6. Good directories for clients to use. Podcasting really took fire when good directories emerged.

(on a sidenote: AMP is evil and publishers are shooting themselves in the foot. It seriosly risks to become an "iTunes" moment, where an entire industry relinquishes control of their market because they don't understand long-term consequences.)


Rest of your suggestions are quite thoughtful, but

> They can license feeds to specific clients only

this is a bad idea.


I still use it every day. I run a python script through Cron that parses a list of feeds and caches new items in a sqlite database. Then it pulls the most recent 100 items out of the database and generates a static html page in which I read them


This is what I've been telling my grandmother to do, but for some reason she keeps using Facebook instead.


I recommend your grandmother try feedly if she finds rolling her own too daunting.


Consider working email in there somehow.


Sending oneself some nifty HTML email would certainly create some cringe, if one likes that.


Why's that?


Why's what?

And probably also: Are you aware that the request to work in email somehow is a joke with about 90% certainty?


Might be useful to some: i realized some time ago (yeah it took me a while...) that most websites that don't have a RSS feed do have a Facebook page. So now i just add those Facebook pages as RSS feeds in my reader and i can follow with RSS those websites that don't have a RSS feed. At least Facebook is useful now.... :-)


How do you do this? I did a bit of research and it looks like FB have deprecated their old RSS feeds based on page `id`.


rss-bridge supports facebook profiles/pages[0].

[0]: https://github.com/RSS-Bridge/rss-bridge


well i guess there are other ways but as i built a RSS reader, i've added this feature. But you're right they deprecated the api, so it only works for public pages, and not people's profiles.


Ok, so you use their JSON API with an FB app token?


well you don't need to login to access these pages, they are public, i ended up doing some html parsing :-)


Makes sense


>That little tangerine bubble has become a wistful symbol of defiance against a centralized web increasingly controlled by a handful of corporations, ...

Just a quibble about terminology. Those handful of corporations do not control the web; they merely dominate it. It's an important distinction. Actual control is censorship, which is an important issue these days. The fact that a lot of people are using systems associated with a few companies is in the end relatively unimportant.

RSS is quite important simply because it is used to discover individual articles from lots of different sources. People who post things with RSS feeds are not raging against the dark but instead are routinely maintaining the light.


> After all, The Onion is a publication that relies on syndication through Facebook and Twitter

https://www.theonion.com/rss

That didn't take long to find. Why use that as an example?


I think you misunderstood the point there. Facebook and Twitter is what drives traffic for The Onion. They can and do syndicate via RSS. But it doesn't help them reach most of their readers.


From a user’s standpoint, who cares? Just because most people don’t use RSS, doesn’t really matter. Every continuously updated news site and blogging site still supports it - as well as HN.

Once I start finding sites that don’t support RSS, then I worry.


I mean, care about whatever you want to. I find it odd that someone who doesn't care posts in an RSS-related thread, but maybe you post about things you don't care about all of the time.

But if you care about what content is available, if you care about how people who create work you enjoy are able to make a living about it, if you care about things like who's the president of the US. Then it might behoove you to care about the fact that between them, Google, Twitter and Facebook control the majority of the referral traffic for the most popular 100 websites. A future where RSS prevailed over social media as a way of curating content for most people is a different world than the one we live in, and if you care about the world you live in, you might care about why RSS didn't prevail, even if it's still usable for the minority who prefer it.


I mean, care about whatever you want to. I find it odd that someone who doesn't care posts in an RSS-related thread, but maybe you post about things you don't care about all of the time.

Reread my post. There is a difference between caring about whether the majority of users are using RSS and caring about whether publishers are creating RSS feeds.

I specifically said “Once I start finding sites that don’t support RSS then I worry.”


Like almost every article about RSS it seems more concerned with people writing than it is with the needs of the readers. That was always the problem with David Weiner and his pals.

For instance what the exact format is does not matter. Supporting RSS, Atom and a few similar formats from a reader perspective is just no problem compared to parsing bad HTML, dealing with JS Hell, etc.

The real issue is a lack of innovation on the part of feed readers, particularly that we still see the "here is a list of all your feeds" viewed separately and no effort to make a workflow where you can make it so your time is not wasted by articles from Tedium and sensationalistic sites like Fox News and the New York Times, etc.

see

http://ontology2.com/essays/HackerNewsForHackers/ http://ontology2.com/essays/ClassifyingHackerNewsArticles/

the machine learning technology to do this is ten years old and if the kids out there weren't so interested in word embeddings, functional programming and other fads, they could bring this to the people.


I tried using twitter to get news before. But the news magazines I subscribed to on twitter tweeted out the same article more than once. It was rather frustrating.


Also nowadays they manipulate what you see. From one source you like you might only get 1 out of 100 posts into your stream, and another source you don't care so much about you get 10.


Inoreader is my main source of gathering news. Social Medias are IMO ill-equiped for this tasks, mainly drowning useful info in a ton of noise.


Another tragedy is that RSS dropped in popularity at the same time that appreciation for the "open" web did. The great fake-news scare has greenlit walled gardens of the kind people never envisioned back when RSS started.

Content discovery for the everyday internet user consists of either Facebook, Google, or Twitter. And none of them are opaque about their SERPS / timelines.


I miss Apples native RSS readers (Safari and Mail used to have native readers, with Mail being configurable to send you RSS feeds to your inbox). Sadly they dropped them with Lion if I'm not mistaken.

RSS is/was nice to be able to stay up-to-date with infrequently updating blogs, and it's honestly a shame that it's no longer used as much as it used to.

One thing that could possibly re-ignite the use of RSS feeds is a button in the adress bar (I recall Safari having this option) to indicate that the site has an RSS feed and taking you to it when you click on it. Maybe look in a couple of predetermined places to find this feed (feed.xml and rss.xml are almost universally the common feed names I've seen used on the sites that I frequent, with the exception of reddit, which simply lets you append .rss to any URL).


MS Outlook still has a feed reader, although it is pretty limited.


what? no one told me!

I still use it every day, its a vital part of my controlling what info I see every day process...


Sadly, the article glosses over what a mess RSS 2.0 was. There were actually three different, slightly incompatible releases of 2.0. It's not really relevant anymore because so many years have passed, but it used to be that in order to parse an RSS 2.0 feed, you'd have to know when the feed was created and parse it differently depending on the creation time.

I read a post about the contentious history of RSS several years ago, similar to this one, and that bit really jumped out at me. I'll see if I can find it again.

Edit: Somebody else posted it here already. https://news.ycombinator.com/item?id=18004137


Fundamentally, RSS is supposed to be a tool to deal with information overflow yet it failed.

The demand is how to get the content of what you want and when you want in the quickest and easiest way.

Social networks, Tweets, FB, RSS subscriber, flipboard etc. all attempted despite the Ad revenue must be taken care along the way, yet none has really met the demand.

Do you get news from your friends, or subscribe people, or visiting sites? To me it is as much as mundane as 20 years ago when internet started. The problem has been resolved except many have made tons of money while tackling it. The fundamental problem remains unresolved and still waiting for answers.


Good "history" post that anyone who do not know should read as personal IT history culture however their conclusion IMVHO are wrong.

When RSS concept started we simply do not have always-on tech, ads-tech, aggregator's tech and software itself was a business. Now that we have the above TheBig simply prefer control their consumers better. RSS-web-client like Google reader are simply ancestor of modern aggregator, a way to control users while leaving a bit of freedom, aggregator's suppress this last bit of freedom, that's for me the real reason behind RSS/Atom decline.

Sorry for my poor English.


I'll say it: I miss RSS.


How do you “miss it”? It hasn’t gone anywhere.


Shameless plug backing up the fact it hasn't gone anywhere: https://draft.li/blog/2016/03/21/rss-usage-on-the-web/


Amount of websites that offer RSS != Amount of users who still use RSS


As long as RSS is available and not being abandoned by publishers, why do you care about the number of users?


What's the best iOS RSS reader for iPad (split view support) and iPhone, which manages a local-only database of RSS subscriptions, without requiring iCloud or other cloud-based login and surveillance?


Reeder is free since v4 will launch later this year. It works fine on Mac and iOS. You can use local RSS or things like Feedly. I just use local RSS. No syncing between Mac and iOS btw.

http://reederapp.com

edit: There is also the news that NetNewsWire is back to its original developer and will be released later this year: http://ranchero.com/netnewswire/


iCloud has surveillance now? Interesting.


https://money.cnn.com/2016/02/22/technology/apple-privacy-ic...

> Apple's stance on privacy and security applies only if you don't back up your data to iCloud ... In the first half of 2015, police agencies around the globe asked to explore 4,472 Apple customer accounts, according to the company. Apple disclosed data to police on 1,886 them, of which 1,407 were provided to law enforcement in the United States. In these instances, Apple gave investigators contents of customer iCloud, iTunes and Game Center accounts. Apple didn't always turn over the information. It objected 401 times in all.


This is no more surveillance than police coming to your house with a search warrant is though.


Occupants of a house are usually aware when a warrant is served.

"Exploration" of iCloud accounts is remote, e.g. ongoing surveillance in search of data to justify a warrant. It can be done by law enforcement from multiple countries globally, while searching of a house is performed by local law enforcement operating under local law.


This is really good, this story of RSS ends up being a representative illustrative example (synecdoche?) of a couple interesting themes in "what's happened with the internet".


I never saw social networks as a replacement to RSS/Atom feeds. Feeds were about checking news/articles directly from a client, whereas social networks are about sharing a link.


Still using it everyday:)


As a developer of a feed reader that supports both RSS/Atom and Twitter/Facebook/Google+/Instagram/VK (https://bazqux.com) I say that making client for well defined API hosted by one provider (silo) was much more simple than support RSS/Atom feeds.

Here is an incomplete list of various problems you'll have if you want to write a feed reader http://inessential.com/2013/03/18/brians_stupid_feed_tricks

It requires a lot of work to make a feed reader that supports all possible feed generators from all possible sites. And it's more related to multiple buggy producers than to multiple feed formats (for example repeating item IDs, different item IDs for the same items, missing IDs -- no matter what format you use you get same issues).

Another complexity is feed updating itself. Silo hosts content and knows what was updated. Feed readers don't. So they need to poll and update a lot of content even if they have just a few users (but with thousand feeds each). There is a PubSubHubbub, but again, lots of providers and implementations could be buggy (for most feeds I just fetch them directly when receiving a push instead of use push data itself since it's more reliable).

So the open nature of web syndication itself dampens its usage by making developing and running feed reader a costly task. It's really simpler (and maybe more profitable if you're lucky) to write a silo.

And there are more problems that are not related to feed format at all: relative complexity of subscribing, not everybody need syndication (many people prefer occasional visiting of the site instead of reading each article), new forms of "syndication" (related videos, algorithmic feeds), less blogs (maybe it's good -- let people post cat pics in social media), less popularity (not a part of big sites, browsers), less education (social media and messaging apps is the only Internet some people know) and so on.

But there are people who want to follow every article from interesting sources in software optimized for this task. And feed readers are the best thing here. For example, some people use my feed reader for Facebook/Twitter feeds only (no RSS feeds at all) just because it's more convenient to read them here.

In the end, I think, process of reading subscriptions is more important than underlying format. If RSS will die (quite unlikely) another methods of getting articles will appear.


still use RSS via newsblur, hope this never goes away


I still use RSS to power my little podcast syndication platform.

While I was building it I did notice it had become harder to find a poscasts RSS feed but they definitely all still used it. I suspect it is used more as a business to business/platform to platform transport these days.

Certainly beats building thirty API integrations for people who want to roll their own syndication for their podcast.


You can find the rss feeds of (almost) all podcasts in the world here: https://www.listennotes.com/

Or use the API: https://www.listennotes.com/api/


Awesome! It looks like you are the creator, I have some questions regarding licensing and copyright I have had trouble finding the answer to.

As the episodes are syndicated it seems reasonable to "rebroadcast" the episode on your site, but the terms and conditions for many podcasts (example 99 Percent Invisible) seem to suggest that is breaching their usage rights.

Have you looked into this at all and did you find similar issues?


I was planning to build something similar to your app, but you built exactly what I was looking for.

How is the audience feedback so far?


So far so good :)

June, 2018: https://www.producthunt.com/posts/listen-notes-3-0

Dec, 2017: https://news.ycombinator.com/item?id=15825900

But one comment that I heard again and again is: It's meaningless to build a podcast search engine, because Google and Apple can easily build something similar :)


What a well written piece! Thank you for that!


I have been using RSS for 12 years. It has been transformed by big players, Is n't it Facebook a RSS reader with social features? I only unsubscribed my friends rss and connected in facebook, other sources like news, releases etc still I am using RSS.


I use RSS for machine to machine syndication. It still is a real simple and brilliant way to scrap content in an easy way. For me RSS is part of the open web.


The title article is missing "and re-rise". I use it regularly and I am happy how it gets the job done : no social stuff, no ads.


I just read this post via RSS in Feedbin. Still very much alive and kicking.

Also... that website proudly displays an RSS feed button on every page.


I'm subscribing to this blog out of spite.


The irony is that I saw this headline using an RSS reader. That's how I read the Hacker News stories.


RSS is a nice format, but it'd be even nicer as json.

Recently had a client that needed me to read their RSS feed into email templates and update their template after they switched their email service. First time working with RSS. 10/10, would do so again.


> [RSS would] be even nicer as json.

Already a standard! https://jsonfeed.org/


I recently added support for JSON feed in my feed reader and blog. After playing with it for a while, I doubt it will be a wildly supported standard.

- It is really just RSS/Atom wrapped in JSON. Sure it changes a few names, makes some elements mandatory and other elements optional but overall it does not attempt to fix core problems (consistent item IDs to avoid reruns, embedding messy HTML with relative links...).

- Besides a dozen of JSON feeds mentioned in the documentation or in the Github repo, there are very few feeds published in the wild.

- Its authors seem to have kind of given up already, a lot of open issues and PRs but not much happening.

Don't get me wrong, I'd like a successor to come and fix the issues that RSS and Atom have but this successor would need much more traction than what JSON feed has.


Oh neat! Thank you for the heads up on this!


I am reading HN through RSS, what demise?


Missing some important discussion topics such as PubSubHubub and WebSub. As the Web-based ad industry continues to crumble under the ideal of privacy as a basic human right my vision brings Web feeds back into the limelight but not as a metadata description frameworks but as monetized content upgrades.


We need to write some software that generates low-quality articles like this and then submits them to hn automatically. Candidates:

. $x is dead

. the rise and fall of $x

. new battery breakthrough claims to $x

. Ask HN: $question_where_the_answer_is_my_own_business

. <Google|Apple> <unveils|retires> create_rand_product_name($technology)

. Study reveals random_food() <increases|decreases> risk of random_quality()

. <any string of english words that contains "Machine Learning" or "AI">

. A random_quality() RayTracer in random_tech()

. ??? suggestions welcome

This would then force hn to have some sort of filtering/moderation.

Randy's law 247: The easiest way to fix a non-trivial system is to abuse it.


Or, just let the system work as it has very well for many years: https://news.ycombinator.com/item?id=18003343


This article is quite a poor example of the issue you are referring to - it's quite a detailed look at the technical history of RSS - but I love your regex spam queries.

Ask HN: $question_where_the_answer_is_my_own_business is my personal favourite.


They changed the hn title at least once. It was "rise and demise". Go here [0] and click on past:

[0] https://news.ycombinator.com/item?id=18002503


Understood - that's also the name of the actual article, although on reflection this article is a bit weirder then your average click bait.

Normally a rise and fall, etc, is only a title looking for eyeballs - what's most odd about this otherwise well researched article is that the last two paragraphs actually proclaim RSS's death, when this is clearly not the case. It's not as prominent without a flagship corporate app like google reader, as noted by the author, but it's nearly everywhere and there are a large number of very usable readers on mobile.

It is a good argument against corporate walled gardens, however.


With the EU parliament's recent voting in favour of requiring consent/royalties for "rich linking" (eg. previewing content without sending the user to the original site), I can see a renaissance for RSS as an accepted and practical means for federated personal news aggregation. But then for me and many others, RSS has never gone away.

While we are at it, we should also bring back Usenet/nntp (though obviously I have nothing against news aggregators with discussion boards such as hn). But long term, only a new distributed p2p web will be able to escape the ever-increasing shilling and censorship. You can start this process right now and be prepared for the future by separating content strictly from web apps and avoiding JavaScript-heavy approaches for content.


> With the EU parliament's recent voting in favour of requiring consent/royalties for "rich linking" (eg. previewing content without sending the user to the original site), I can see a renaissance for RSS as an accepted and practical means for federated personal news aggregation.

That's funny, I see the same legislation as the death of all unauthorized RSS aggregation. The "link tax" provision isn't about links, it's about excerpts of content.


But with RSS you're receiving content directly from the publisher's site. There's no third-party involved, and the publisher can provide as many text bites as desired. There is no problem there as long as no content aggregation web site is acting as middleman.


> as long as no content aggregation web site is acting as middleman.

That would imply people will go back to installing things, hunting for feeds, copypasting urls... As much as I like desktop tech, that boat has likely sailed, at least for the mass-consumer market. You might maybe get away with a mobile app, but you'd still need help from browsers to "subscribe" to feeds - something that was pretty awful the first time it was tried, and has been basically abandoned or removed since.


> That would imply people will go back to installing things, hunting for feeds, copypasting urls...

Nope. We could have a simple list of feeds, with a simple 'add' button next to the site name, even embedded in the app itself, as preferred/most read feeds list. Since this list contains no excerpts, and actual excerpts and feeds would be coming from publisher themselves, it shouldn't be an issue.

Of course, then we have questions like order of this list, how many and which feeds it lists and all, but it offers enough convenience that if made avoidable, we shouldn't have a problem with it.

Or since IANAL, this _may not_ be how the links and excerpts are considered, and I'm wrong, in which case, I'd like to know so I can correct myself.


We can invent new, mobile-friendly protocols for discovery, especially if existing methods become difficult under new laws.


I was going to say "you can invent anything you want, but good luck convincing "Appoogle" to help you", but in truth, they've actually let the door open on mobile with their support for custom urls, so you are right there.

On the desktop it would be harder though.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: