Infinite scroll is also a way to make scrolling infuriating, especially if you are wanting to click links and fan out from that location: it forces you to avoid your browser's carefully designed history stack like the plague (opening new tabs or windows for navigation instead) lest you lose your incredibly valuable document you've carefully been scrolling down in (and which is now so large it is threatening to cripple your browser anyway). What happened to normal "next page" paradigms? Does user testing really show that infinite scroll is better?
That's not a problem inherent to infinite scroll at all. It's easily fixed by setting a hash value on the current page (the #foo in a URL) corresponding to the current scroll state, and updating it each time infinite scroll happens.
Why Twitter doesn't implement something like this is a whole different issue.
You can imagine attempting to support something like this when the user has scrolled 20 pages worth through a timeline. We could, of course, backfill newer tweets subsequently up to the most recent (taking a minimum of two requests and a maximum of 20), but this would be expensive.
I mentioned somewhere else in this thread that we could maintain an accurate scroll height but detach more recent tweets when a timeline is scrolled a long distance. We could do the same thing here, and backfill as the user scrolls up. There are some challenging things to get right though if the user decides to jump to the top, etc. Definitely non-trivial.
Thanks for the response, I thought the reality would be much more complicated due to issues of scale.
This is primarily about supporting the back button, and I don't think users expect the latest data when going back through their browser history - generally pages aren't refreshed, and that's not an unusual or unexpected behavior. So you could cache each page state, without worrying about displaying newer tweets, though the caching mechanism itself incurs some nontrivial cost.
