Interesting article, though I think the rationale (find an algorithmic engine to identify quality content) a bit forced. Solving that problem seems similar to building an AI capable of interpreting what a human being would think. Far simpler is to use ratings people already have (book reviews, links to sources they like, forum discussions, up/down votes, etc.) and highlight those.
For example, you could look at the top 100 hacker news links each month and get a large amount of insight/interesting stories/. I think people enjoy browsing, discovering and voting -- I'm not sure I'd really want a machine that just gave me the most interesting links (hn services as that, with the added gem of perhaps discovering something off the beaten path).
I did like the part about focusing on insight, and calling that out as a specific trait. I find that is one of the few transferrable skills -- facts, while interesting, are fragile; ways of thinking about the world can be more permanent.
"Solving that problem seems similar to building an AI capable of interpreting what a human being would think. Far simpler is to use ratings people already have (book reviews, links to sources they like, forum discussions, up/down votes, etc.) and highlight those."
Being right 100% of the time would require some AI that's way more advanced than anything existing today, but the thing is that you really only need to be right 25% of the time to create something that's an order of magnitude better than what exists today. For example, let's say you have this article: http://www.alternet.org/drugs/21673/
Being able to identify a set of people, 25% of whom would find that article worth reading, shouldn't be very difficult. In addition to all of the rating systems you mention, you can also do some basic things like tracking their reading history, asking them some questions before and/or after presenting them with article recommendations, etc.
In fact, the two things I would most go out of my way to avoid would be A) trying to figure out what people will like based on what other people with similar taste like and B) pulling specific facts out of articles and trying to analyze them or compare them with other articles. These two approaches seem like incredibly hard problems to solve, and they don't seem at all necessary to create something that's an order of magnitude better than what we have today.
The trick is getting humans to do as much of the work as possible. For example, one could create a delicious-like tagging system whereby people would tag articles with other articles they should read if they liked/disliked or agreed with or disagreed with the original article. Once you have a shared vocabulary to talk about the problem, getting a good solution isn't that hard even if not all the tools are 100% formalized.
In fact, the two things I would most go out of my way to avoid would be A) trying to figure out what people will like based on what other people with similar taste like and B) pulling specific facts out of articles and trying to analyze them or compare them with other articles. These two approaches seem like incredibly hard problems to solve, and they don't seem at all necessary to create something that's an order of magnitude better than what we have today.
How is that delicious-like tagging system different than trying to figure out what people will like based on what other people with similar tastes like? It seems that a recommendation system with sufficient input would easily be able to identify the 25% of people whom would find an article worth reading.
Here's the thing, I don't care about what the vast majority of people think is interesting, insightful, funny, or informative, because I've already obtained those insights. I want to hear what people whom I respect think is cool, and that is actually the principal value of Twitter for me. Tweets from my favorite friends, and people whose mind I respect, go straight to my phone. I want to be alerted with that information ASAP.
A long piece but one of the few I've read in full lately - this is a great article and if you're interested in ideas and semi-formal concepts relating to communicating ideas, I heartily recommend it.
I'm not knocking hacker news at all. My point though was that let's say you only want to read articles that are counterintuitive, contrary to your current beliefs, contain facts that you don't already know, etc. With enough computing power and a little human mediation this should be doable, but it's not something you can get from sites like HN currently.
Overall, I agree with you. It should be possible to define some objective criteria about what is interesting, and then some algorithm could check all the new articles from mainstream dailies, weeklies and monthlies, and return let's say up to 150 articles monthly which should be interesting. That I think could be an useful service.
For example, Vanity Fair has 1 or 2 articles monthly which are interesting, but I don't have time to read VF every month to see if I discover something good...if some algo can do that pre-selection, that would help.
This gave me an idea: decades ago I became interested in a natural language processing theory known as Conceptual Dependency Theory (never was very useful to me though, and I was like a dinosaur caught in a tar pit: my fascination with the theory kept me in a stuck state).
Anyway, CDT attempts to extract structured information from text and I thought that if CDT worked, then you could measure something Alex talked about in the article: rating text as interesting if it introduced novel concepts.
One issue with this article is that while a majority of big ideas get committed to paper at some point; writing is just one medium in which to transmit ideas.
I like the emphasis behind the article, but we need to get off this idea that writing is the thing, when really its ideas that matter.
Awesome article man. I was actually thinking of the humor dichotomy in the shower this morning (things are either funny cuz it's true, or funny cuz they're not true) but then I concluded that my definition was stupid because everything is either true or not true.
For example, you could look at the top 100 hacker news links each month and get a large amount of insight/interesting stories/. I think people enjoy browsing, discovering and voting -- I'm not sure I'd really want a machine that just gave me the most interesting links (hn services as that, with the added gem of perhaps discovering something off the beaten path).
I did like the part about focusing on insight, and calling that out as a specific trait. I find that is one of the few transferrable skills -- facts, while interesting, are fragile; ways of thinking about the world can be more permanent.