As a second-career middle and high school math teacher, I have a working theory.
In previous eras in the US, we taught primarily procedure and facts, and assigned lots of practice work. The average kid did _all_ the practice work, for societal reasons that have eroded but are still present in other cultures.
In the course of grappling with all the practice work, the human brain couldn't help but recognize patterns and start to make broader conceptual connections, which led to deep understanding.
Today in the US, teaching facts and procedure first doesn't work, because very few kids get enough practice to start to draw deeper connections. So we are teaching conceptual understanding first, and then layering procedure on top.
But I don't think this is worse. There is some research showing that it works better than the alternatives, and in my experience the top 10% of students (the ones who would have learned well the old way) are still doing quite well and honestly just getting to the "math is fun and interesting" part of the journey a lot earlier in their school careers.
I have been doing some work on some lawsuit stuff recently, and I have been told repeatedly that the average juror is absolutely less intellectually capable than they used to be. In the 90's, the writing level to use for expert reports was a 6th-7th grade level. Now, if you are a court expert, you should be writing and speaking at a ~3rd grade level, and I have been recently told that even this level seems to be beyond the average juror.
This is also information from a group of people who are highly incentivized not to lie to you. Unlike school officials who are incentivized to say that students are doing better than they are and clout-seeking education researchers, the question here is how to speak persuasively, and there is no judgment (well, they are lawyers, there is equal contempt for everyone). They also do enough science (mock juries, polling, etc.) to get a decently accurate picture beyond the level of "anecdata."
While the top students are doing fine, they honestly always will do fine. The bottom 90% of students is doing worse in terms of actual education that makes it to their adult life in the current educational model than they were doing before. Whether that is due to a culture shift or a change to new supposedly-evidence-based education methods is not clear to me, but it is very clear that outcomes from schools are getting notably worse.
The way I estimate the situation (and I admit this is not a rigorous scientific conjecture), is the following (for public schools in the US in the average):
Early years: bad pedagogy, bad retention rates (ie, quitting after 5th grade to go work on the farm, bad average results, basically only the top 10% learned deeply and went on to intellectual pursuits
1900s - 1980s: Decent and improving pedagogy (the aforementioned procedure-first style for math), good and increasing retention rates, good parental and societal pressures to perform, great average results, top 50% or more went on to intellectual pursuits.
1980s - 2010s: Same math pedagogy, but with rapidly deteriorating parental pressure to perform, leading to worse results. A truly terrible detour for reading instruction (from phonics to context-based reading) that decimated the average reading level of those currently under 40. Currently being fixed but not yet replaced in all schools. See: "the science of reading".
2020 - 2024: an earnest effort, gaining traction and fast-tracked after the educational disaster that was COVID, to find curricula that actually work with current students.
The concept-first, teach-them-how to think approach for math really is pretty new, and only just now being rolled out in a lot of states.
In reality, a vanishingly-small subset of American students has ever been given an entire education using evidence-based instruction and curricula. Looking at what actually works and trying to synthesize it and scale it up state- or nationwide is truly a brand new experiment, and one the decentralized US education system is sort of designed to prevent. So we'll see.
I understand that you are optimistic about evidence-based instruction and curricula in math, but the disastrous reading curriculum you have discussed was rolled out with much the same "scientific" study and fanfare as "evidence-based" teaching is today. As an example, take a look at all the science that surrounded "whole language" teaching of English. The skepticism you see of "common core math" and a whole new set of math teaching techniques is somewhat rooted in the experience of the same sort of thing happening through the last 20-50 years with mixed results.
As a society, I honestly think we are a little too hooked on scientism. Not science itself (which we don't do nearly enough of), but treating the output of scientific research like a religion. In the few pieces of educational science that are actually proper blind studies, the p-values are abysmal. It's worse than psychology. And yet, every ~10 years, "the science" gives us a new way of doing things that turns out not to be any better than the old way (and plus it's new so nobody knows how to do it). As it turns out, "the science" in education usually means "a small cohort of very good, very enthusiastic teachers tried this, and their outcomes were better than their peers." I assume you can see the problem with that. There is no "phase 2" trial of this stuff or anything controlled, just a rollout of a new method.
In previous eras in the US, we taught primarily procedure and facts, and assigned lots of practice work. The average kid did _all_ the practice work, for societal reasons that have eroded but are still present in other cultures.
In the course of grappling with all the practice work, the human brain couldn't help but recognize patterns and start to make broader conceptual connections, which led to deep understanding.
Today in the US, teaching facts and procedure first doesn't work, because very few kids get enough practice to start to draw deeper connections. So we are teaching conceptual understanding first, and then layering procedure on top.
But I don't think this is worse. There is some research showing that it works better than the alternatives, and in my experience the top 10% of students (the ones who would have learned well the old way) are still doing quite well and honestly just getting to the "math is fun and interesting" part of the journey a lot earlier in their school careers.