I would think the more relevant number is year-to-year increase benchmarked to the median increase across major US metropolitan areas. You're right though: median wealth and income were probably also growing much more commensurately mid-to-late 20th century than they do now.
