Show HN: gofeed, a robust RSS and Atom Parser for Go

[drakenot]

https://github.com/mmcdole/gofeed

[pkulak]

Wow, this is great. I built a web scraper in Go a while back that had feed support, and this is exactly what you end up writing. You need the title, description, items, etc, but you really don't care if it's RSS or ATOM. So you translate all that crazy into some common format before you start working with it.

I love how comprehensive this is. I didn't even know a lot of the ways you list to get the different RSS properties. :D

[Jemaclus]

This is pretty rad. Is there a way to customize this for non-RSS feeds, or is this pretty much tailored for RSS/Atom feeds?

[drakenot → Jemaclus]

What is another feed format you'd like to see supported?

If you have some custom thing you could take a look goxpp[0] which is the XML Pull Parser I wrote to use in gofeed.

[0] https://github.com/mmcdole/goxpp

[code_research → drakenot]

The selfoss reader [0] has a nice concept of extending the format of the incoming data feeds, would be also a very nice feature for your code. Even nicer it would be if new incoming data formats could be configured in yaml files without having to recompile.

the selfoss author mentions these kind of data:

* RSS Feeds * Images from a RSS Feed * Images from deviantArt Users * Images from tumblr * Your twitter timeline * Tweets of a twitter user * heise News with full content * golem News with full content * MMOSpy News with full content * RSS Feeds with readability

certainly not the most important sources, but it gives an idea about how diverse data sources are on the net today, so limiting to rss and atom is not a good idea. maybe some people would like to add facebook streams or other "social" sources.

Also please take a look at the extensive testing suite of feedparser [1] - lots of feeds there, that will break any naive assumption about how clean real life data will be.

[0] http://selfoss.aditu.de/ [1] https://github.com/kurtmckee/feedparser/tree/develop/tests

[drakenot → code_research]

I've actually already ported hundreds of test cases from Mark Pilgrim's feedparser.

[code_research → drakenot]

ah, that is great! I think feedparser has the biggest collection of obscure feeds, it is an awesome collection of what to expect in the wild. If you know some more sources of weird feeds produced by "modern web software", please tell me, thanks!

[Jemaclus → drakenot]

I deal with a lot of custom feeds, so not a lot of standard formats. I'll take a look at goxpp! Thank you!

[dzdt]

I read it as "goofed" at first.