Sure, but what you're describing is the difficulty of learning as age sets in. That's very different from the original assertion- "If I start specializing elsewhere I'll forget the intricate details of the thing I'm already specialized in." - unless that person was describing themselves as older.
Perhaps fixing degradation will prove to be a less difficult task than increasing memory, as the growth of new neurons is still a very newly-studied phenomenon compared to treatment into neurodegenerative processes, and soon medical science will benefit the learning of all demographics.