People have a weird habit of treating Hacker News like a glorified CMS, when in fact it's a community with a distinct CMS. The CMS is actually quite awful for mobile or touch-based viewports, so Hacker News sometimes works in spite of its CMS.
That's why these things die, because there's some assumption that using the Hacker News CMS will somehow build an active community on its own.
We're seeing the same thing going on with all the forum/comment CMS being touted recently, but people are giving the CMS way too much credit for the - hypothetical - success of a community-based site.
You can still argue that the arrangement of a particular CMS will be conducive to certain types of discussion over others. E.G. The adversarial type of discussion we see here would be far less effective without threads.
For hackers, this makes perfect sense since every "fork", if you will, needs to be followed to the conclusion of its job. Threads ensure ancillary points are confirmed, refuted, altered or somehow interacted with.
Likewise, having the reply form page show the parent's message content ensures quick reference to the points (quoting, another hacker favorite, becomes easier).
In a way, the layout and functionality of a CMS is communication feng-shui.
I personally prefer forums (linear, non-nested comments) to what is on Hacker News, but people disagree vehemently on this, so that will be litigated for years to come.
Ironically, pg has implemented measures to prevent deeply threaded discussions, so he seems to be somewhat aware of innate disadvantages to using this format of discussion.
I think that, by and large, CMSes at best can serve to remove obstacles to creating content and having discussions. It still falls on the community and admins to nurture a culture and atmosphere that resulting in said content and discussions.
Generally, though, there are nested/threaded comments (HN, reddit, Disqus), and there are linear (forums), and between the different CMSes in each categories, we're overstating the differences vastly, especially with regards to advantages and disadvantages. In other words, types of CMS can have a huge difference, but individual CMSes of the same type don't differ as much as the people touting them would like us to believe.
Places like NeoGAF and Something Awful are practically held together with scotch tape, but they're still amazing communities in their own right.
What would be nice is being able to choose between nested or flat layout, and to have folding. I actually like the "expand" feature Livejournal and other forums currently use which just appends the next level onto the current thread... though all of this would probably require javascript which a lot of HN users wouldn't really like.
Still, I think the more users can customize their own experience, the less likely they'll be to complain about it.
The biggest change I think could be made would be to lock the display order of comments based on submission time alone.
Hacker News never really has so many comments that you can't read through them all in a minute, and if a discussion would grow too large, because it's a threaded discussion system, you might as well give up discussing the story anyway. Just look at "scandal stories" like those related to Heroku and such.
Because Hacker News is karma-based as any other comment CMS, I would be more inclined to write my comment as a reply to the top-ranking comment instead of posting a top-level comment myself, since I'm likely to gain more karma from this. This completely destroys the flow of discussion.
Comment CMSes will always be gamed by people, but this would probably be the most significant improvement to the quality and linearity of discussions.
Options for viewing a discussion differently don't work very well, because then people start discussing in wildly different ways, which makes for a huge mess. That's why, again, a linear/forum CMSes are superior when it comes to discussions, although they don't lend themselves perfectly to a link/self-oriented submission system.
That's why these things die, because there's some assumption that using the Hacker News CMS will somehow build an active community on its own.
We're seeing the same thing going on with all the forum/comment CMS being touted recently, but people are giving the CMS way too much credit for the - hypothetical - success of a community-based site.