One thing that is becoming more and more clear, is that memorization is absolutely a required part of becoming an expert.
But it's not fun, it's boring, and it takes up a lot of your time. You can't really train your memory with a nice Youtube video, and it's incompatible with the "learn while playing" concept.
So schools in the US tend to sideline it by handwaving that "they teach how to think" instead. And I think we're now seeing the result, with the ever-declining test results. The NAEP scores in the US peaked in 2014 and have been declining ever since: https://www.nationsreportcard.gov/ushistory/results/scores/
It's also interesting that there's no similar decline in China, where rote memorization is unavoidable because of the writing system.
As a second-career middle and high school math teacher, I have a working theory.
In previous eras in the US, we taught primarily procedure and facts, and assigned lots of practice work. The average kid did _all_ the practice work, for societal reasons that have eroded but are still present in other cultures.
In the course of grappling with all the practice work, the human brain couldn't help but recognize patterns and start to make broader conceptual connections, which led to deep understanding.
Today in the US, teaching facts and procedure first doesn't work, because very few kids get enough practice to start to draw deeper connections. So we are teaching conceptual understanding first, and then layering procedure on top.
But I don't think this is worse. There is some research showing that it works better than the alternatives, and in my experience the top 10% of students (the ones who would have learned well the old way) are still doing quite well and honestly just getting to the "math is fun and interesting" part of the journey a lot earlier in their school careers.
I have been doing some work on some lawsuit stuff recently, and I have been told repeatedly that the average juror is absolutely less intellectually capable than they used to be. In the 90's, the writing level to use for expert reports was a 6th-7th grade level. Now, if you are a court expert, you should be writing and speaking at a ~3rd grade level, and I have been recently told that even this level seems to be beyond the average juror.
This is also information from a group of people who are highly incentivized not to lie to you. Unlike school officials who are incentivized to say that students are doing better than they are and clout-seeking education researchers, the question here is how to speak persuasively, and there is no judgment (well, they are lawyers, there is equal contempt for everyone). They also do enough science (mock juries, polling, etc.) to get a decently accurate picture beyond the level of "anecdata."
While the top students are doing fine, they honestly always will do fine. The bottom 90% of students is doing worse in terms of actual education that makes it to their adult life in the current educational model than they were doing before. Whether that is due to a culture shift or a change to new supposedly-evidence-based education methods is not clear to me, but it is very clear that outcomes from schools are getting notably worse.
The way I estimate the situation (and I admit this is not a rigorous scientific conjecture), is the following (for public schools in the US in the average):
Early years: bad pedagogy, bad retention rates (ie, quitting after 5th grade to go work on the farm, bad average results, basically only the top 10% learned deeply and went on to intellectual pursuits
1900s - 1980s: Decent and improving pedagogy (the aforementioned procedure-first style for math), good and increasing retention rates, good parental and societal pressures to perform, great average results, top 50% or more went on to intellectual pursuits.
1980s - 2010s: Same math pedagogy, but with rapidly deteriorating parental pressure to perform, leading to worse results. A truly terrible detour for reading instruction (from phonics to context-based reading) that decimated the average reading level of those currently under 40. Currently being fixed but not yet replaced in all schools. See: "the science of reading".
2020 - 2024: an earnest effort, gaining traction and fast-tracked after the educational disaster that was COVID, to find curricula that actually work with current students.
The concept-first, teach-them-how to think approach for math really is pretty new, and only just now being rolled out in a lot of states.
In reality, a vanishingly-small subset of American students has ever been given an entire education using evidence-based instruction and curricula. Looking at what actually works and trying to synthesize it and scale it up state- or nationwide is truly a brand new experiment, and one the decentralized US education system is sort of designed to prevent. So we'll see.
I understand that you are optimistic about evidence-based instruction and curricula in math, but the disastrous reading curriculum you have discussed was rolled out with much the same "scientific" study and fanfare as "evidence-based" teaching is today. As an example, take a look at all the science that surrounded "whole language" teaching of English. The skepticism you see of "common core math" and a whole new set of math teaching techniques is somewhat rooted in the experience of the same sort of thing happening through the last 20-50 years with mixed results.
As a society, I honestly think we are a little too hooked on scientism. Not science itself (which we don't do nearly enough of), but treating the output of scientific research like a religion. In the few pieces of educational science that are actually proper blind studies, the p-values are abysmal. It's worse than psychology. And yet, every ~10 years, "the science" gives us a new way of doing things that turns out not to be any better than the old way (and plus it's new so nobody knows how to do it). As it turns out, "the science" in education usually means "a small cohort of very good, very enthusiastic teachers tried this, and their outcomes were better than their peers." I assume you can see the problem with that. There is no "phase 2" trial of this stuff or anything controlled, just a rollout of a new method.
I think you're wrong to be honest. Experts have a lot "memorised" but never do memorisation. Chess grandmasters can recall huge sequences of moves from classic games. It's not because they sat down and tried to memorise "1. e4 e5; 2. Nf3..." It's because they understand those games, why the moves were made, why seemingly obvious alternatives weren't made etc etc
I have a PhD in mathematics and some (minimal) claim to expertise. I have also met and worked with several serious experts. What I said about chess holds true here, at least in my opinion.
Every strong chess player I know actually has sat down and memorized the lines in their repertoire. It's also true that they understand the "why" behind the moves but I don't think the memorization is avoidable at a high level. It's not always possible to find tactical compensation over the board, especially when you're running lines that have been tested by an engine, which many people are!
Or for instance see the "woodpecker method", which is basically a technique for rote memorization of tactical patterns used by a lot of strong players.
(I quit my PhD in mathematics so I have less claim to expertise, but I regularly get smoked by >2000 fide chess players in my local club and talk to them about chess a lot)
This is kind of conjecture based on various things I've read over the years, but I feel like there is a strong tie-in between memorization and visualization abilities. I think many great chess players actually have trained their visualization abilities and are great at this. This in turn allows for easier memorization of games, sequences, etc, because it feels like they're only half-memorizing it at this point. It's easier. Some great Chess players utilize the memory palace technique as well, which feels like it ties into their visualization skill abilities.
I think there is often a confusion between memorising and understanding because they often go together.
I'm an amateur chess player and I learned a few lines from a few openings. I never sat down repeating blindly the move to make them stick in my head, I watched lessons that explain every move. But I did that on many openings and understood most. Yet I can only play a few of them because I forgot all the ones I was not actively practicing.
Understanding makes remembering a lot easier but it isn't enough.
> I watched lessons that explain every move. But I did that on many openings and understood most.
People who say this usually didn't understood the lesson at all. When you ask people about the things directly afterwards they can't answer a thing, its just that people overestimate how much they did understand when they first heard it since they don't didn't test themselves on it.
Everyone I know who is 1700 or above has spent a significant amount of time memorizing openings from books, and it would be interesting to see a counterpoint.
> One thing that is becoming more and more clear, is that memorization is absolutely a required part of becoming an expert.
Has this ever been in question?
I don't think any serious expert in ML that is pushing against LLMs is making a claim that memorization and/or compression isn't a necessary part of intelligence. Rather that there's more to it.
Too much memorization is a bad thing, it's called over fitting. Bringing up schools is a good example. I'm sure many here have met people who can answer questions really well when in specific contexts but not in others. People who do well on tests but not in the lab. The difficulty of word problems is a meme, but are just generalization.
If you ask me, what makes humans and animal brains special is the fuzziness. It's this seemingly contradictory nature of well defined understanding through rules (such as physics) but also understanding that resolution is far from perfect. In our quest to become more precise it is recognizing the impossibility of precision and finding balance.
What I'd say is wrong with both the US and China is failing to teach how to think. The truth is that this is exceptionally difficult to test, if not impossible. It's difficult to distinguish from memorization when the questions are not clearly novel. But how do you continually generate sufficiently novel questions when not teaching at bleeding edge?
> Too much memorization is a bad thing, it's called over fitting.
It's not. It's just a skill that might become unused, like playing piano. Children who grow up in religious cults that emphasize memorizing the holy texts (Hasidic Jews, some Muslim sects) do surprisingly well on standardized tests. Even though they receive a fraction of instruction time.
> What I'd say is wrong with both the US and China is failing to teach how to think.
And I maintain that you can't learn how to think without grinding through facts, learning how to organize them in your mind.
This article does not appear to be supporting a point counter to what I said. It is also focused on the opinions on non-experts. In fact, the majority of the article is discussing how schools aren't "following the data."
I would be surprised if the dominant method was "memorize" for reading, as this would mean a curriculum that has little free reading.
I'm not too interested in what non-experts have to say unless there is quite compelling evidence. The average person quire frequently overestimates their confidence in how something should be done.
> It's not. It's just a skill that might become unused, like playing piano
I grew up playing piano and memorizing holy texts. Even participating in scripture competitions as well as music competitions. With the highest confidence I can assure you that no professional in either of these subjects believes that one should memorize without limit. In music they will use the words "without soul" while in scriptures they may say that you know the words but not the meanings.
I'll directly quote from the bible to demonstrate both at once:
> Therefore I speak to them in parables, because seeing they do not see, and hearing they do not hear, nor do they understand.
Mathew 13:13[0]
I suggest reading the chapter in full, as it makes the point more explicitly.
I think both groups understand something important to language (which yes, I will argue that music is _a_ language): that the "words" (sounds) used are only tools to convey what is the deeper meaning inside. In music you seek to draw that out of the listener. In scriptures it is the same. The clearest cases of these may be proverbs, parables, koans, or fables. The words hold only what is at the surface. It is ironic you specifically mention Hasidic Jews, as they are deeply entrenched in the Kabbalah, which is famous for being entrenched mysticism. That there are hidden meanings in the scriptures. This isn't even uncommon in religion in general! I don't think it is hard to see this in music or any art. If you are in any doubt, please go visit your local art gallery and listen to one of the local artists. Even if you believe they are full of hogwash, it still illustrates that they are trying to convey something deeper. If you wish to get this lesson and learn a bit about Jewish mysticism at the same time I'd recommend A Serious Man[1] (a Coen brothers movie)
I must stress that language (of any form) has three key aspects: what is intended to be conveyed, the words and way the words are used (diction), and the way the person receiving interprets this. The goal is to align the first with the last, but there is clearly a lossy encoding and lossy decoding.
> I maintain that you can't learn how to think without grinding through facts
I'm not sure why you thought we were in disagreement. Perhaps you know the words but not the meaning. I hope your head does not feel too heavy from all the things you carry in your mind[2]
> This article does not appear to be supporting a point counter to what I said. It is also focused on the opinions on non-experts. In fact, the majority of the article is discussing how schools aren't "following the data."
In this case, "experts" who thought that "learning by playing" is better were wrong, as proven by data. Schools that use the traditional memorization-heavy approach of learning letter combinations do better.
The example is Oakland's schools, where teachers considered the traditional approach to be "colonizing". Test scores cratered as a result.
> I'm not too interested in what non-experts have to say unless there is quite compelling evidence. The average person quire frequently overestimates their confidence in how something should be done.
If you want to map the political system on more than one axis, you could look at economically liberal/conservative and socially liberal/conservative separately, but in a two-party system, not all quadrants will be equally represented. This particularly annoys libertarians.
If you want to map education politics on more than one axis, one axis could be progressive/conservative in the political sense (Are we 'woke'? Do we teach that some people are gay? Let kids pick their own pronouns? Do we have prayer in school? Base our values on the Bible? etc. etc.).
The other axis is 'progressive'/traditional in terms of methods, e.g. do facts matter and should kids learn by rote?
Again, not all quadrants are equally represented. The 'progressive' side in terms of methods genuinely holds that rote memorization is bad and useless, and even taeching kids to read with phonics is outdated - where that led, is explained in https://features.apmreports.org/sold-a-story/ which has been featured on HN before now too.
Unfortunately, most attempts to debate the methods axis online degenerate into fights about the political axis (see also: California math reform).
> What I'd say is wrong with both the US and China is failing to teach how to think.
The problem is, I'd say, that there is no such thing as "how to think". Thinking about environmental policy is very different from thinking about debugging **ing JavaScript callbacks (sorry, having a bad day). You need different degrees, for a start! There will never be a way to teach social scientists generically "how to think" so they can just pick up programming in 30 days, because all you need to do is apply your thinking skills to code! In the same way, we can't just "teach how to think" in a CS class and then expect the graduates to double up as MDs in a pinch, because that's just thinking about the human body!
You can only do "critical thinking" in an area where you have domain knowledge. That's one reason that memorization is required - you need the base of domain-specific facts so you have something to think with.
Other responses are pointing toward this position, but I want to make it explicit:
Memorizing an elegant algorithm is a lot more fun and more broadly useful than memorizing the look-up table for a limited section of the results; but sometimes caching parts of the look-up table in your memory is necessary for expertise.
For example: Many bright kids get frustrated when they have to repeat a section because of calculation errors, when they already get the concept. Even if you know a few good algorithms for multiplication, memorizing "the times tables" for double-digit numbers can be useful.
On the other hand, you don't have to memorize everything. If you can reliably re-derive some result from theorems you know well, and you never need that result in a hurry, it's fine to leave it un-memorized.
I'm not sure how you can link these test results to lack of memorization when so many other factors are likely at play from the worsening parental support many children face with school due to economic decline, to the pittance teachers are paid, to the war on education some states seem to be playing (curiously they are all heavy red states).
The Red states are actually staying fairly steady.
I don't think that the lack of memorization is the main reason, but it definitely is one of the reasons. Mobile phones and tablets in education are also not helping, and their spread is well-correlated with the start of the decline.
I'm not exactly sure what you're arguing here. It more seems like you're pointing to California in 4th grade. Which I'm not sure I'd feel comfortable using California as the prime example given how diverse it is in culture and demographics. Too much noise introduced through aggregation.
When looking at the other grades and varying by subjects it very much looks like the data is noisy. I'm sure there's meaningful analysis to be drawn from here but I'm certain there's no obvious one.
You can look at other states and at the 8th grade. It's the same pattern almost everywhere: achievements peaked in 2014 and have been declining. CA, WA, NY were hit pretty severely.
I like to look at 4th grade because most of the learning by that grade _is_ memorization.
> One thing that is becoming more and more clear, is that memorization is absolutely a required part of becoming an expert.
I'm very interested in this idea, but it doesn't immediately ring true to me, is there some research or other stories you know of that can escalate this from "interesting" to "must do"?
I agree with almost all of that, including that there's no similar decline in China. I wonder whether it's really "because of the writing system" though. As a counter-argument:
- We may have fewer individual symbols (letters) to memorise, but we still need to learn vocabulary in English.
- In European languages with grammatical gender you have to memorise that too (in German, a fork is feminine but a spoon is masculine), same for languages with declensions.
- And when you study something, you have all the technical terms in your field.
- But you don't simply memorise (if you're clever) what inode, tail recursion, ACID, syscall etc. mean, you subconsciously construct a knowledge graph.
- Chinese characters have structures and relationships too so you don't just have to memorise each one individually. For example, the character for "man" (as in "male") is a combination of person+field, I believe; whereas the word for "business" is apparently the compound buy+sell. So the "knowledge graph" argument still applies in China.
- Children in school in China don't need to know all the charaters, they start off with a useful subset and learn (or look up in dictionaries) more as they go along.
I actually learned both English and Mandarin Chinese :)
Vocabulary in English is a problem, but it's much easier to learn than thousands of Han characters. Grammatical gender is not a problem for native speakers, you already know it from speaking the language.
> - Chinese characters have structures and relationships too
Most Han characters can be decomposed into simpler characters. But not all, and you still have to memorize that a character for "branch" (支）consists of "ten" (十）and "again" （又）. And of course, some characters have multiple meanings, and even multiple readings.
In short, you have to memorize a lot.
> - Children in school in China don't need to know all the charaters, they start off with a useful subset and learn (or look up in dictionaries) more as they go along.
Of course. But they still have to learn at least 2-3 thousand by the end of the school.
> it's incompatible with the "learn while playing" concept.
That's definitely not always true.
Even boring things can be turned into a game. Different games are fun to different people.
> schools in the US tend to sideline [memorization]
US education incentives are far more closely aligned with teaching for standardized test than "teaching how to think", whatever their claimed rhetoric. It's been that way since even before W and the NCLB Act, which made things much worse.
Are you sure we're not conflating memorization with repetition, at least in certain domains? These debates focus a lot on early childhood education and core skills like math and literacy – but I think for, say, math competencies expected at an 8th-grade level (which these NAEP scores focus on) like ordering fractions or working with linear equations, I'm not sure "memorization" is more applicable than "repetition" (and, in any case, memorization isn't a significant subgoal – e.g. formulae or rules for determining concavity etc. where remembering them isn't comparable to bulk memorization.)
You could certainly argue that this is more about foundational skills like reading and writing, but it does limit the scope of how we're thinking about things.
(Personal caveat: I was taught elementary school math using a "reform curriculum", so my memory might be faulty.)
> it's incompatible with the "learn while playing" concept
I'm not 100% sure about that. I do believe "learn while play" is less efficient for memorization, and it especially sucks when it comes to somewhat hard-to-see patterns. But you still do get memorization from it. (E.g. in music, you could either learn the circle of fifths, or you spend forever noodling on songs and at some point discover that relationship).
Learn-while-playing is essentially ab initio research, and that only works for an inquisitive mind, with lots of friction added over memorization.
I'd suspect that something like structured play is striking the best balance between being interesting and being useful. It definitely works for kids, based on the research I've read. I'm less clear if it works for adults, but... probably yes? Just nobody's building those structured opportunities?
There is a difference between memorization and rote memorization. In chess, rote memorization of master games or chess positions is not a recognized training method. Chess memory improves as a byproduct of analyzing many positions.
Rote memorization is absolutely a mainstay of learning both openings
and endgames.
It’s usually a part of tactics training as well although not as purely, the polgar sisters for instance were drilled on the same chess positions day in day out in a spaced repetition system. This is going away a bit because chess puzzle databases have so many unique positions that there’s less need for repetition.
Regarding openings, there's a trade-off between chess training and chess results. Rote memorization can improve your results (if you already have good skills), but it won't improve your skills.
Learning endgames is not about blindly memorizing moves in specific positions. You learn tricks that can be used in a large number of positions. Even the seemingly very specific positions can be mirrored left-right (not to mention black-white).
That seems reasonable, but at the same time my understanding is that there’s enormous value in novice and intermediate players to memorizing openings. I wonder if that effect is significant enough to categorize chess as another high-rote-memorisation-affinity task.
Learning openings beyond a very basic level is not going to help the club player very much and it’s generally a good way for them to waste their time, at least from an improving your ELO perspective.
Being the best out of the opening will typically put you a “quarter pawn” ahead, maybe putting you ahead as white or equalizing as black. Then if you’re a novice you will immediately hang a knight and end up 2.75 pawns behind. Then your opponent will hang a bishop and you’ll be a quarter pawn ahead again.
The other problem with learning opening theory against novices is you will learn 30 moves a side of Ruy Lopez opening theory and your opponent won’t get 10 moves without leaving theory rendering your study moot.
There’s far more emphasis on memorizing openings at the grandmaster level because people are playing a tight enough game elsewhere for that slight advantage to really matter, and because of all the pre-game preperation where teams of grandmasters and chess engines will come up with novel moves to throw an opponent off balance while the star player memorizes the lines. To the point of grandmasters like Bobby Fischer complain it ruined the game and inventing variants like chess960. All super grandmasters have outlier memorization abilities.
Generally club players just need to rote memorize not too deeply and understand the broad sweeping ideas and key moves of the openings (when white does that, counter them with this). That should allow them to come up with reasonable moves on the fly which might be the best or third best moves. Memorizing fewer openings at first is probably better. At the more casual level memory is much less important.
> Being the best out of the opening will typically put you a “quarter pawn” ahead, maybe putting you ahead as white or equalizing as black. Then if you’re a novice you will immediately hang a knight and end up 2.75 pawns behind. Then your opponent will hang a bishop and you’ll be a quarter pawn ahead again.
While this is true if you know openings, many openings have a trap or two that make up a very tricky line that puts you 3-5 points ahead. Knowing the traps and how to punish them is a huge material advantage in some games. So while knowing your opening well is "worth" only a quarter pawn in a typical game, it is worth a several-percent increase in win rate from knowing these lines.
Openings like the Jobava London system have 10-20 different trap lines like this, and if you want to play them, you must know the lines.
It is very common for players with your mindset to plateau around 1400-1600, at which point it's time to sit down and start memorizing openings and endgames. Just being good at searching the game tree gets you to that point, but now you need to know the times when the game tree collapses 30 moves later.
There was a guy Michael De La Maza who literally just drilled tactics and broke 2000 USCF and then quit chess, and if you look at his games yes he really really did not understand openings. So 1400-1600 is well before when you’re going to plateau without knowing openings.
1400 yes learning a trap line can improve your results, so if you subscribe to the Eric Rosen school of opening theory you can benefit from openings. I’ve just never thought it’s worth learning much about conventional openings until about 1600.
> When all is said and done, I can’t recommend Rapid Chess Improvement (a book that, in my view, offers a philosophically bankrupt vision of what chess is). It smacks of "the blind leading the blind.” But, as I said earlier, his book might prove useful for some.
Also, a rating improvement from a 1300 start after a long spell of no rated games often means a lot of skill improvement in that gap, and then a corresponding adjustment in rating. Perhaps the guy was a bridge or Magic: the Gathering player and already had a decent intuition for games and needed to transfer that to chess. Disregarding that drilling 1000 tactical problems sounds a lot like a memorization plan to me, he also clearly knows the e4 opening given the game analysis quoted in Silman's review.
> Like many adults, he assumed that he needed to augment his natural skills and intelligence by compiling chess knowledge: he studied openings, endgames, and other "chess knowledge" information. Despite all that accumulation of knowledge, he was getting nowhere.
Huh... did someone study some openings and endgames? His tactical game was likely the weakest part of his game so he remedied that error and got rapid improvement. Not in spite of failing to study openings and endgames, but because he did study them, just out of order.
Sure he didn't know the quarter-pawn-advantage grandmaster lines (which you don't need to know as a 1600), but he knew the traps and how to avoid them.
This can't be more wrong. It's absolutely a training method, and the importance of recognizing certain openings is even more pronounced in professional play.
That is not correct as some of the comments suggest.
Typically experts are designated as persons whose mastery of a craft extends beyond memorization such that they have forgotten the specifics of things learned as confidence and muscle memory have replaced memory recall. They are at one end of the Dunning-Kruger paradigm in that they have forgotten things learned, or avoided things unnecessary or anti-pattern, resulting in gaps of explanation that are not present at time of performance.
It is beginners that are reliant upon memorization because they have not yet achieved the practiced mastery sufficient to perform with immediacy otherwise.
No. If you're willing to not be a fake expert, and wait until you've actually learned the material before holding yourself out as an expert, you don't need to rote memorize.
But it's not fun, it's boring, and it takes up a lot of your time. You can't really train your memory with a nice Youtube video, and it's incompatible with the "learn while playing" concept.
So schools in the US tend to sideline it by handwaving that "they teach how to think" instead. And I think we're now seeing the result, with the ever-declining test results. The NAEP scores in the US peaked in 2014 and have been declining ever since: https://www.nationsreportcard.gov/ushistory/results/scores/
It's also interesting that there's no similar decline in China, where rote memorization is unavoidable because of the writing system.