Actually since 4.something you can enable "auto-updates", and it will keep itself up to date.
I wouldn't suggest keeping in client machine on an older release, much less on a 2,3,4 or 5 year old release as you suggest, and leaving it in 2.3.3 until now is downright criminal with all the security fixes that have come out since then.
Wordpress ecosystem has many security issues. So upgrading doesn't mean it's secure. So I run my wordpress on laptop, compile all things to static contents thanks to httrack , and publish those static contents. No more nightmare, and the speed is amazing.
Of course, there is no more comments and dynamic contents. But I don't care ;)
I wouldn't suggest keeping in client machine on an older release, much less on a 2,3,4 or 5 year old release as you suggest, and leaving it in 2.3.3 until now is downright criminal with all the security fixes that have come out since then.