It did begin then. The Supreme Court of the US is since then conservative and will now probably remain so for many years to come.
Further, he needed the second term then, so he couldn't go all crazy as he needed the people to vote for him once more. Now he doesn't have that limitation any more.
Things take time. Erosion of trust and the creation of political apathy in the populace takes time. Also, as has been said, he did try things but was continuously pushed back on by the actual politicians he put in his cabinet. He's also 8 years more elderly and emboldened. His cult of personality has essentially stabilized into an American institution. He's also had an entire administration to place judges and pass legislation that favors his power plays. In general it seems like you're asking, why might it take more than 4 years to topple a democracy, which i think has an obvious answer, democracy doesn't want to be toppled.
Again, I'm not arguing he's gonna go full dictator, but i think it's a lot more likely this time around than last time.
Further, he needed the second term then, so he couldn't go all crazy as he needed the people to vote for him once more. Now he doesn't have that limitation any more.