I've been running DupDetector for just under a week now. The results have been interesting. Many, many duplicates have been found, and many, many more items have been cross-referenced. So often the same story is reported again and again from different sources, and while this isn't a bad thing in itself, my personal belief is that the resulting divided discussions is a waste of time and effort. And wasted time is something I hate.
I come to HN for high-quality links, and even more high-quality discussion, and anything that dilutes that discussion is, to me, a bad thing.
I originally intended to run the DupDetector for a week, but after 5 1/2 days there's enough information to tell me what I want. The thing that has caught me by surprise is the way it has had such widely differeing reactions. Some people have accused me of "a novelty account", something I'd never heard of, but which appears to be associated with Reddit. I thought of it more as a robot assistant.
In particular, I thought more people would be interested in the technology and the hacking. More than anything it's the lack of a response on that level that's made me pause.
And the final factor is today, when DupDetector's karma has fallen from 27 to 12. I don't care about the karma, but it's an indication of people's feeling about the exercise.
I'll run it for just a few hours longer as I tidy things up, but basically I'm stopping, explaining, and I'll see what the response is. I thought it would be, and it would be thought to be, cool, interesting and useful.
Maybe I've misjudged my audience. I am reviewing the situation.
I hate them. Why? Because the entire point of social bookmarking is to find things you find interesting, not things that you find unique. That little arrow to the left of the title means "I found this link interesting. I think other people will find it interesting as well."
It absolutely does not mean "This link is unique. Nobody has seen it before." If that were the point, we could just pipe RSS feeds into the URL submitter, couldn't we?
The very fact that links are appearing on the front page means that a lot of people haven't seen them yet. It means that they got some utility out of reading them, and it means that they thought others would too.
Sometimes, dupes are good. I don't remember who said it, but that Louis CK interview that gets posted every once in a while, where he is talking about how we're surrounded by wonderful technology and yet nobody cares, they said something to the effect of "I wouldn't care if this was stickied to the top of the page and everybody had to watch it every single day before they post. He is making an excellent point."
Now, I think this is a bit excessive, but the point stands. It isn't about being unique, it is about being good.