The perfect thing of Wordpress is that it really helps to upgrade from (very) older version to the new one.
Yah, some months ago I upgraded a wordpress 2.3.3 (released in 2008, http://codex.wordpress.org/Version_2.3.3) to version 4.1 :) I only needed to fix some minor problems!
I will wait 5 years from now to see if I can move from this 4.1 to another one. Sorry Wordpress 4.3 ;)
Actually since 4.something you can enable "auto-updates", and it will keep itself up to date.
I wouldn't suggest keeping in client machine on an older release, much less on a 2,3,4 or 5 year old release as you suggest, and leaving it in 2.3.3 until now is downright criminal with all the security fixes that have come out since then.
Wordpress ecosystem has many security issues. So upgrading doesn't mean it's secure. So I run my wordpress on laptop, compile all things to static contents thanks to httrack , and publish those static contents. No more nightmare, and the speed is amazing.
Of course, there is no more comments and dynamic contents. But I don't care ;)
Yah, some months ago I upgraded a wordpress 2.3.3 (released in 2008, http://codex.wordpress.org/Version_2.3.3) to version 4.1 :) I only needed to fix some minor problems!
I will wait 5 years from now to see if I can move from this 4.1 to another one. Sorry Wordpress 4.3 ;)