Back in the 80s and 90s ARMs were very popular in the US because interest rates were high, and the expectation was that they wouldn't stay that way forever. The rate increases had a lifetime cap, so there was a knowable upper bound.
Edit to add: also, if you knew you were going to sell (and confident you could actually sell) within a few years, the lower initial rate of an ARM made sense.
There is also a material spread between ARMs and fixed when the rates are high. When rates are low the spread compresses (same with rates between terms 15/20/30), making the additional risk questionable.
I have a buddy who got a fixed 15y when rates were super low b/c he wanted to pay his mortgage off early. But, locking into the 15y barely lowered the rate vs a 30 fixed. I told him to do the 30 fixed, and just pay it as a 15y. This would give him flexibility if he lost his job or had some other emergency.
Getting a 30y and paying off in 15 is a smart move. For our most recent house, that's we did set out to do, until the last refi (maybe 5 years in to the 30y) when we got a 15y and paid it off in 10 more years.
I used to think that, but i'm not convinced anymore. In the best case it is smart, but I've known too many people over the years that either died before they were able to enjoy the fruits of their investment, or by the time they were that old their body was such that they couldn't do anything without much pain.
Better to pay off the house when you retire and enjoy life a little more. Of course you should save for a nice retirement, but don't plan all for when you get old. (Renting can also work out well, but you need more in other investments when you retire so you can keep paying rent)
You bring up a great and different point. Is paying off early worthwhile at all? I agree with you that it's really not. Given a low fixed rate, paying off a mortgage early is almost never a good financial decision. Particularly with a lump sum. It's most people's largest leveraged investment. Better to keep the cash and the leverage working for you.
Edit to add: also, if you knew you were going to sell (and confident you could actually sell) within a few years, the lower initial rate of an ARM made sense.