In today's world, all of the information for personalization has to be tracked on someone's central server. That's a lot of data per user. If you have a lot of users, that doesn't scale very well. And thanks to the stateless nature of the web, it is natural to have to process that data on every single hit.
Historically with Usenet, all of the information for personalization was distributed along with the data. The information for personalization was generally much less than the data for Usenet. And the processing of the data set was generally done once per forum per Usenet session, followed by much viewing and reading of data.
The result is that when sites like Slashdot popularized web forums, tracking that kind of detailed individual reading history didn't seem like a critical feature. If you've got one or two webservers serving a semi-popular forum (I believe that HN runs off of a single CPU), that feature just isn't going to be seen as a critical need.
I do understand that the current web infrastructure is not made for this, but given demand, it /could/ be adapted for the sort of reading that we are nostalgic about. There are obscure forums that I sometimes read on the portal bsh.co.il - this is an Israeli social work portal. Click on a message in a forum like http://www.bsh.co.il/forums/AllMessages2.asp?Fnumber=10 - there is going to be a checkmark next to the message you clicked from now on, even if you reload the browser (presumably, there is a cookie). This is a small custom-written forum with very little programmer muscle behind it, I doubt it's more than a couple of guys. If they can do it, everyone can do it - and it's a small step to hiding threads completely, etc., etc.
I've concluded that there is just no demand for sophisticated threading/reading software of the type we had in the '90s. If there was demand, there is certainly technology to fill it. If they can run Linux in a browser complete with a little C compiler, they can run the equivalent of trn. As you pointed out yourself, Gmail covers the basics already - why not make it more sophisticated and bring the technology to Google Groups? No demand. Otherwise, one of the sources of innovation in this industry - big corps, startups, or academic/open source, would have filled it. Even if it required extending HTTP/HTML somehow - these extensions happen quickly when really needed.
The features that you are talking about are essential for carrying on extended, complex conversations in a high volume environment.
The high volume discussion forums that we have today are mostly geared towards providing and filtering discussions of topical items for short periods of time, while blocking spam. (Not a big problem on the classic Usenet.)
The result is that we have things like voting systems, but the implicit assumption that any old conversation is automatically dead. (An assumption that is implicitly guaranteed by the fact that the forum becomes unusable.) So that leads to lack of motivation to improve software for this use case.
Yeah, tell me about it - I poked in here more or less by accident (tab still open) and found your reply. Everyone else is long gone, of course.
You're right about topicality being important, but there are tons of forums which I've followed (on history, for example, or on science) where nothing is topical and threads go on for 140 phpBB pages or more. Needless to say, these threads are unusable, especially compared to Usenet. So why not make software for it? People write software for more obscure niches all the time.
In today's world, all of the information for personalization has to be tracked on someone's central server. That's a lot of data per user. If you have a lot of users, that doesn't scale very well. And thanks to the stateless nature of the web, it is natural to have to process that data on every single hit.
Historically with Usenet, all of the information for personalization was distributed along with the data. The information for personalization was generally much less than the data for Usenet. And the processing of the data set was generally done once per forum per Usenet session, followed by much viewing and reading of data.
The result is that when sites like Slashdot popularized web forums, tracking that kind of detailed individual reading history didn't seem like a critical feature. If you've got one or two webservers serving a semi-popular forum (I believe that HN runs off of a single CPU), that feature just isn't going to be seen as a critical need.