Then how do you have all that money? And where are you keeping it?
(1) Earning that money is still growing the economy, and (2) even a zero-interest bank account is growing the economy (you're just not seeing any remuneration for your part in that) - and (3) even if you took the money home and destroyed it, since you're still working, you're providing a very useful deflationary function since it means someone else can print more money safely (and once again you're helping grow the economy, just yielding your output to a third party).
Eventually you work less so you never earn that money in the first place. This may involve saving up first so you have more economic bargaining power, or it may involve collective action to reduce working hours across the board.
My only point in referencing saving is that it is one path to achieve the bargaining power to actually work less, against employers who inherently want to occupy your entire life. I'm talking about having saved up enough to be comfortable working less - after which you can turn your reduced spending into reduced working. An alternative avenue that doesn't involve saving at all is collective action, which is how we've gotten down to a 40 hours per week standard.
"The economy" is defined by what is legible in the financial system. In this example, that is working for a wage. Time spent on other endeavors is not reflected in such accounting (sure, money spent on those activities is obviously still part of the economy, but said money is only part of what is spent on them). Time spent on other endeavors is therefore not "growing the economy", except in some pathological sense of trying to shoehorn our entire existence into economic terms.
Time spent with family or on hobbies certainly does make society better though, despite not being economic activity or "growth". In fact, this is exactly what opponents of the growth-worship philosophy are aiming for.
It's identical to if you destroyed it. It just adds some risk that you might suddenly mobilize it (hence one of the reasons it's actually illegal for individuals to store large amounts of physical cash for long durations).
Also irrelevant anyway from point 1: you're still working to make money, thus you're growing the economy.