What would be cool is if you implemented something like Netflix, you know like a recommendation engine. Something that bases of your tastes and preferences and also uses the preferences of those users that have similar tastes as you do. Base on that you can recommend what's "worth reading." :)
And you can figure out what people have read using the (soon to be closed by Mozilla) CSS history hack. You can easily index every story on HN by pinging and scraping newest every 20 or minutes. Don't worry they wont block you ;)
You can grab that from searchyc.com, or just ignore it. You can get reasonable recommendations based on the latest articles. This would not be a general purpose recommendation engine, so you would actually need much less data. Attempting to analyze 1 million+ articles seems like overkill in this case. You could just scrape for a week. If you are building this I actually have several months worth of articles indexed, and would be happy to provide the db dump of 80,000+ articles.
I'm actually thinking about hacking this up this weekend as well.
Not being facetious, just trying to find out if this site is for me: all the summarized articles so far have zero technical content (ie stuff on Python, Javascript, memcached, hardcore bit-on-bit action, etc). Is this intentional and indicative of hnsummary's content in the future?
Now this is me being facetious: should the answer to the above be yes, I find both hnsummary's name and its tagline of "Just the stuff worth reading from Hacker News" to be presumptuous at best, misleading and condescending at worst.
You should disclose what you consider to be "worth reading". Hopefully you have a bias that is different then the front page, otherwise it's really no different than that.
On the other hand, he is contributing both a news filter and summaries. Perhaps there is value in the summaries, even if the filter mirrored the front page directly (sort of like Slashdot?).
Cool. Can we get the story on this? Is it done by a human, e.g. will it stop working for a week while said editor is on vacation? Is it auto-generated by vote count? Details please.
[edit] I have to assume it's an editorial thing, since the summaries couldn't be auto-selected like that, I don't think.
Yes human. I summarize articles I find most interesting and would be of interest to the HN community as a whole. As long as I'm up-voted then I'll keep doing it.
I've thought about that. If I see interesting comments I may work it into the summary with a citation. My first choice would be to direct people back to HN for the article discussion.
How so? Why would the most popular stuff always be the stuff worth reading? That's certainly not true for most media. Do you suppose that the Billboard Top 100 is the music most worth listening to?
Not saying that having an editor pick them out is necessarily going to do a better job, but there's certainly room to do a better job.
I think in a relatively niche community where the readership submits the content and then determines what's popular, ranking probably correlates better with quality vs billboard or other mass media where the relationship is strictly producer -> consumer. That's just my intuition though.
In any democratic system, the actual result of taking a vote will be a minimax of sorts: the result with the least aggregate distaste. This is why, if you have a non-niche demography for a vote, the top results will always be completely bland—no one can find a reason to dislike them, so the few people that do vote them up push them "all the way" up. Meanwhile, the contentious items—the ones that some love, and some hate—end up getting pulled down, because those who dislike it dislike it enough to overcome the friction of moving the mouse to the down-vote arrow and clicking (or to going out and putting out propaganda posters against that party, etc. etc.).
It would be a genuine invention, perhaps one of the only things worthy of an "algorithm patent", if someone were to come up with a way to encourage creative focus through the use of democracy. Imagine if "designed by committee" were actually a good thing!
As it stands, democratic voting systems are only useful as trained classification filters (i.e. spam filters where the "spam" is "stuff we don't like"). Actually ranking the "good" results should probably be left to editors.
Well, I agree, of course, it's hard to say that we don't do better than Billboard. That was a little bit of hyperbole. But consider: People upvote things for a zillion reasons.
Because it made them chuckle.
Because they agree with it.
Because it shoots down another submission they really disagree with.
Because the post wasn't very good but it had a link and the link was really good.
Because the author came and posted in the comment thread and said something interesting about it.
Because their buddy posted it.
Because a minor HN celebrity posted it.
Because it's all about banana-flavored threadsafe data structures in Erlang and they did their Ph.D. thesis on banana-flavored data structures and nobody ever posts about them anymore.
Most of those barely correlate to what we mean when we say "quality" at all. The people who upvote only things that are "high quality" probably upvote much less, in fact, than people who upvote for other reasons. So that's why I would not expect upvotes to correlate with quality.
At the moment, everyone upvotes anything containing the word 'iPad'. So boring. I think editorial control usually works far better than voting up/down by users. Democracy absolutely sucks.
There's a whole lot of "why I like/don't like the iPad" stories that seem to poke holes in your theory. Even in niche communities, the (relative) junk can worm its way to the top.
EDIT: Didn't notice axod's post deeper in the thread with the same point. Upvoted him.
This is an awesome idea and could be awesomer if turned into a full news website instead of just a blog, with multiple editorials and pictures and shit.
Maybe at some point you should allow users to write their own reviews so that your admins aren't stuck writing them all the time? Maybe some sort of social networking utility?
What's the algorithm for what makes it to this site? Or is it some random guy's editorial discretion?