You can't downgrade/upgrade the OS to a specific version of an update after a fortnight or thereabouts. That's an artificial constraint applied by Apple. It's not even possible to do so offline with iTunes even if one has the IPSWs. If you check the /r/jailbreak subreddit, talented coders have to hack the SEP and build complicated, low-level-interacting software like futurestore in order to perform a semi-successful downgrade/upgrade.
However, if you install an update to try it out, or because you didn't realize that it would e.g. break 32bit support, you can never downgrade again (unless you happen to be within a two-week-ish period.)
You don't have to install updates.