Would that be why people raised in dysfunctional or abusive families have very deep rooted issues? Everything learned later in life seems more fluid, but some of those problematic attitudes or personality traits are extremely hard to change. Seems consistent with those parts being build in lower layers with a decades long training experience. And yet at a higher level things can be learned and changed fairly easily.
This is essentially linear algebra, not behavioural psychology. I believe that one shouldn't draw such broad conclusions from a 1% improvement on some evaluation dataset.
I was talking about the ability to use thin layers over top of broad learning. If neural networks are a realistic analogy at all, I think it fits. The roots of all this ML stuff is not straight from linear algebra even though much of the math is.
You're playing it extremely fast and loose with concepts like "low-level prerequisite knowledge" and how exactly does something "rel[y] on that knowledge", though. These aren't physical quantities like temperature where we -- as a species -- have the massive amount of low-level prerequisite knowledge that allows us to make rapid high level judgments that rely on that knowledge. The previous sentence is an example of how easy this reasoning is to abuse.
Could be possible. For example, when a child is deprived of any human and humane contact, they end up with a very very impoverished linguistic system as well as delayed development of all cognitive abilities. Could be because the lack the unsupervised training phase.
