> HN now gets over 120k unique ips on a weekday, and serves over 1.3 million page views.
Still off just one server? What are its specs?
How up to date is the news.arc file in the Arc distribution?
I'm considering using the simpler, more functional, single server, all in memory, filesystem as a database approach in an upcoming project. Would really love to see more details from PG and others who have had success with that back-to-basics simple approach.
Thanks for the info! I hope you don't mind me asking a few more questions...
> The currently available news.arc is quite out of date.
Any chance of getting an updated version?
> Operating out of memory works very well for an application like this, where most requests are for recent stuff.
Do you ever release cached resources? Or does the process simply die with an out-of-memory error, only to be restarted with empty caches?
If the former, what is the cache invalidation strategy?
If the latter, how often does the process die? What are the implications on availability and performance, particularly surrounding process initialization and warming the caches.
We randomly throw older items out of memory. If they're needed again they'll get reloaded from disk, but it's unlikely they'll be needed soon because older stuff is mostly visited by crawlers or random Google traffic.
I read HN in Google Reader in a tabbed browser, and I think I would prefer a feed that reversed the order of the two links. So that the first link goes to the HN comments, and the second link goes to the article itself.
That would allow me to use Google Reader's List view which does not display the expanded, secondary link, and then go down that list of HN articles, open the intriguing HN comments in a background tab, and then when I get to the tab, click on the link in the comments that takes me to the article itself.
At some point I stopped clicking on the links in HN altogether. Instead, I go straight to the comments with an 'open link in new tab' command. If the article is worth reading, I go deeper from there. But the filter comes first.
same goes with their twitter feed. I like seeing the article but I come to HN for the comments as well (come for the articles - stay for the comments ;) )
better yet - I would be cool if they implemented a "reddit bar" for their twitter feed links so this way you can they have HN Bar which goes on top of the article
I'm one of those people who might be perceived as "crawling" against the "More" link.
I use a Firefox extension to pull the next 10 pages, inline. I then convert all the links to open in new tabs. I'll slowly work my way through the resulting Frankenpage.
When I first started doing this, I only pulled 5 pages, sometimes waiting a bit to then pull 5 more/deeper. I was worried about excessive use -- including triggering whatever blocking you have in place. Eventually, I tried 10 at once. I've found that 10 is about as deep as I want to go into the recent history and keeps me fairly current with daily browsing.
I hope this use isn't considered excessive. It's run just once or twice in a day, and the results are simply for personal consumption.
As I (somewhat vaguely) recall, I began doing this when and because a change made the "More" links expire more -- as in, rather -- quickly. That was a time when you were working to keep HN from bogging down and choking on its memory usage. I wanted to see a few pages into the history without a bunch of overlap and having to start over from the first page; and pulling the subsequent pages all at once worked well for me as a solution.
Perhaps my http://hckrnews.com will suit you, since it archives anything that makes it onto the homepage (without crawling against the 'More' link).
I've resisted putting RSS feeds on hckrnews, because I think RSS feeds are a poor match for the format. Number of points and number of comments combine with the link to provide information that you don't get with an RSS feed. Often the utility of a link is not obvious from the source and title only.
Well, I also enjoy finding and promoting good/relevant links that escape attention "amidst the deluge".
I also take your point that the comment links and point counts are pertinent. In particular, I will often start off with the comments rather than the linked page itself. Again, I'll have to look at the revised RSS feed, but as I recall, when I looked at it in the past, I, too, found it less useful for these aspects (unless I was viewing one of the third-party feeds that I seem to recall being created, some time back).
I suppose one could view my use / time on HN as high. OTOH, I pull the pages (i.e. this "Frankenpage") once and then slowly pay attention to them. I guess one could contrast that with another user hitting the top page every 15 minutes throughout the day.
(I do also check the new/ and classic/ front pages a few times, often later in the day. I used to drill further into new/, but I've been trying to limit my time on HN somewhat. Also, new/ is so full of spam, these days, that it's kind of discouraging -- although people need to continue to hit it and aggressively kill off the spam and other crap.)
----
Completely aside, I'd like a comprehensive list of post links. I used to have some old favorites saved, but some of those were lost in a theft. I guess it might not be the healthiest thing for the HN ecosystem including system load, but I'd enjoy journeying back in time to the early days. Many of those discussions were incredibly informative and focused.
I know there are and have been a third party archives and/or views into HN, but either I didn't take the time to learn or the interface itself didn't seem to offer a simple, time-based listing.
I could comment on my sorry little life, but to what point? I'll just reiterate that, technically, the entire pull ends up in one tab. And I close the other tabs as I finish with them, one by one.
With 120k unique IPs a day, it is really surprising how few votes stories get. Most are under 100 points, only about once a month is there one over 1000 points. Do most of the readers not have accounts? Do people with accounts not up vote?
As a long time reader and infrequent commenter, I rarely vote for any stories or comments, here or reddit. When I vote, it's usually on something I find especially interesting or thought provoking.
I rarely vote articles, it's irritating to navigate towards the little arrows: it's enough to target both the article link and comments link. Only if I catch a good article early enough I might give it a vote to boost it's visibility.
For what I've observed, giving a 0.01 implicit vote for each entry where both the article and the comments page were opened should yield sensible results. Articles chosen for reading have already been preliminarily filtered in my brain and I generally enjoy most of those articles.
There was a statistic on Reddit yesterday that 90% of the people on that site did not have an account - and, 90% of the people who had an account have never voted.
Also I think there is a huge drop off in the number of upvotes for a particular story once it gets into the three figures. I only ever upvote small stories because that's when I feel I'm making a difference. Big stories are already big.
I only vote up stories that I would probably consider wanting to come back to again in the future, as a bookmark, since they get added to the user's saved stories list.
Is there anything like alterslash.org for HN in an RSS feed? A link to the article plus the top 5 comments or so would be great to pull in an RSS reader for on-the-go reading.
Is there a way we could also get the total points, submitted by, time submitted and no. of comments ? (essentially the second line). Thanks very much in advance.
I hope this doesn't happen. As a frequent Android/iPad user I have grown to love iHackernews ever since it was published. Unfortunately the lack of comments in most of the posts is really hurting the service and it would be nice to see some collaboration between HN and iHackernews.
I'm already using my custom CSS for HN in Chrome but sadly this isn't possible on mobile and iHackernews fills the need perfectly.
Still off just one server? What are its specs?
How up to date is the news.arc file in the Arc distribution?
I'm considering using the simpler, more functional, single server, all in memory, filesystem as a database approach in an upcoming project. Would really love to see more details from PG and others who have had success with that back-to-basics simple approach.