I feel like spaced repetition is exactly what you shouldn't do. Spaced repetition is great for rote memorization of arbitrary facts but programming is about how to think about and solve problems.
If you master the methods and ideas, you should be able to derive the answers on the spot. That's better than memorization because then you can actually deal with real interviewers who 1) Want to hear your train of thought and 2) Will give you random changes or variants to problems.
It's not about memorizing solutions, because in reality when are you ever going to reverse a linked list or balance a red-black tree? No, so if you're going to put in the effort anyway, you should learn the concepts and understand when to apply them so your knowledge is actually applicable. This will actually make you a better programmer, and you'll do better with the interviews that are actually good (I speak with experience giving interviews at FB).
Learning the concepts and using SRS are not only not mutually exclusive, they're complementary. There's no quicker way to "grok" a deep technical or mathematical insight than to simply expose yourself to it in a diversity of cognitive contexts. SRS works incredibly well because it generates this diversity as a function of regularly scheduled time. Think of it as entropy generation coupled with heavy reinforcement.
Right. I learned what binary trees were in college and didn't directly use them much afterward. Since interviews sometimes ask about trees, I started my practice of solving tree problems by practicing how to invert a binary tree. I practiced it so much that I can invert a binary tree from memory now, so that I won't get bounced from an interview like the inventor of Homebrew was from a Google interview.
Most of the concepts I learned, although I don't remember learning Kadane's algorithm for solving the maximum subarray problem in school, so I've learned some things like that. I can also implement a solution for an associated Leetcode problem in one line of code, so I wound up memorizing that too.
OP says "If you master the methods and ideas, you should be able to derive the answers on the spot", but I hear from a lot of people that does not happen, even (implicitly) in this thread. Any how, even if I use OPs method, if I do the same problem over and over and over again I'm going to wind up memorizing it any how.
Learning is like nutrition. There is no ”one true way", instead there are lots of people and each one has their own approach to learning.
Some people are closer to each other, and upon realizing that they believe they've found a pattern. But it's like when people tell you that you need to supplement x in your diet. It's possible that x helped you. And it's even possible that you'll find other people with the same nutritional deficit, creating the illusion that everyone would benefit from supplementing. But that's just what it is: a premature extrapolation from anecdotes
I was at Google when that happened, and had some inside sources (ie gossip from the guys who interviewed him, who said they didn't even ask any tree questions, and memegen).
I tried to find some outside sources, but nothing about Howell admitting his 'artistic license' outright. So take my fallible memory with a large pinch of salt.
So, inverting is just swapping the left and right pointers for the entire tree? Why would you ever do that? That will flip the ordering of the tree and none of your search or balancing functions will work anymore. Shut up! Don't ask questions! Uh, ok.
I thought we were done with the myth that once we understand something, we don't need to memorize. I've come to the realization that memorization is a core competency. The more I have memorized, whether it's syntax or concepts or problems and how to solve them, the better equipped I am and the more productive I am in the future when faced with even novel problems. Memorization saves me insane amounts of time in day to day programming and gives me an edge in being able to solve and understand problems. I don't just have a deep understanding of the code bases I work with, I have a lot of details memorized about what parts there are, how they work, how they're interconnected, certain technical details, etc. And sure, I could just check, but when having meetings about technical problems and planning our projects, it's an insane help to know so much off the top of my head and that I only need to check a handful of things.
Spaced repetition is nothing other than intentionally refreshing something you currently know, to ensure that in a month or a year or longer, you'll still know it. It's not your place to judge what to memorize or not. I'd suggest you should also use spaced repetition on the methods and ideas, since we're very prone to forgetting that too. I would also never criticize anybody for memorizing something, since it's literally just doing intentionally what we would otherwise do haphazardly and by accident.
This is a common misconception not supported by scientific evidence.
What experimental psychology and neuroscience research has found is that the better people memorize topics, the better they understand and reason about them.
Edit: Of course, you can memorize things without any effort to understand. But if you are making these efforts, having an efficient and effective memorization technique can help a LOT.
One of the things that puzzles me is when I see say Anki flashcards for mathematics things that I never learned like that. I never intentionally learned anything in mathematics in that was and it feels very 'wrong' to me when I see people trying to learn like that. I did a lot of practice looking back, but there was absolutely no conscious memorisation - the nearest I got was at university level when I would sometimes run through the main steps in a long proof before the exams, but it would be at quite a high level when I did.
I don't know if I'm just lucky in terms of how my brain works in some way and I found I had a pretty good 'natural' memory when it came to things like learning languages, and I appreciate that this is totally anecdotal. That book looks interesting however and as somebody who teaches, I'm interested in what works for everybody, not just me.
SRS is not meant as the basis of learning. You study and learn first, then you input what you've learned in SRS, and it'll help you to solidify those learnings and keep them readily available when you need them in practice.
I am always deathly afraid of forgetting a definition/key result in an exam.
Whenever I practice, I always have my notes in front of me and I focus on puzzling through the problem at hand rather than testing my memory.
But in an exam, for me there us always the possibility of failing to recall some fact. To combat this, I record every not-completely-obvious fact into Anki and aggressively practice it in the run-up to the exam.
The multiplication tables and two digit arithmetic are 100% memorization, and more challenging multiplication/arithmetic is just learning to break the problems into memorized chunks and then combine them. Math is 100% memorization.
Spaced repetition is a good way to remember anything, including methods. Just review the method (maybe with a small practice problem) on an SRS-based schedule, to hone your skill. When you encounter an issue in the wild, having committed to memory ways to solve it will speed up the actual execution.
You’re partially right that it’s not about memorizing solutions; it’s about committing patterns to memory.
I well doubled my income after using SRS to master leetcode/algo/ds patterns. Implementing SRS was a catalytic moment in my interview prep training that really accelerated the volume of patterns I held in the RAM of my brain.
I disagree. To be fair, I used to think this too, but then when I started to actually used spaced repetition in the process of learning more complex subjects I became surprised by how much more fluid my reasoning became.
Writing good flashcards is about
- noticing similarities and differences,
- exploring variations of a concept,
- finding generalisations of related concepts,
- learning properties of a concept, etc.
All these things are things you use when reasoning also about more complex ideas.
In other words, spaced repetition is an efficient way to raise the baseline for what counts as a "fundamental fact" which makes it possible to think at a higher level of abstraction.
That’s a fair perspective but consider that spaced-repetition can be an “MVP” of the broader concept of adaptive / personalized learning. You track some state about your knowledge of the world, in this case your performance on a flash card, and optimize the next piece of educational content around it.
It feels obvious to me that, in theory, we could do a lot more to leverage your current world knowledge to recommend the next piece of educational content, to optimize not just your understanding but also other things like how engaged you are. For example, a system could throw you an easier question if it helps you focus longer.
When I see the state of current interview prep it’s mostly “here’s a big list of questions” , perhaps tagged by difficulty and grouped under related topics such as “graph problems” or “tree problems” but I’m personally convinced a more sophisticated system could serve you the best possible next problem to stretch your brain in the right way.
Spaced rep is simply the proven starting point. The rest of adaptive learning has a bit of a troubled past because it was trendy to VCs , but pushed on educators and presented more as an alternative to a teacher rather than something to augment a teacher. I worked for a company that raised 100M+ to work on it but the CEO was great at terms sheets but uninterested in actually building a great education project.
But the reason that I joined the company was the high level idea still resonates with me. Surely many HN users have a big list of things they want to learn- perhaps about LLMs. But a typical course will have zero knowledge on where you’re starting as a student . It might bore you with stuff you know already or take for granted key prerequisites and skip them.
Anyway, I’m currently building my own adaptive learning platform but focused on helping professional poker players learn game theory . My idea being that it’s an easier ed tech app to bootstrap as the knowledge very directly translates to money. And really the same criticism applies because you can’t truly memorize a game tree , it’s more important that you build a high level conceptual understanding. But , as much as rote memorization deserves to be maligned when done in isolation, it’s not so bad when done as part of a broader learning strategy. For example, you can’t memorize vocab to learn Spanish but certainly knowing 5000 words in Spanish is a very nice starting point compared to not knowing any words. And tools like spaced rep have been proven by research to help with that goal and I view as a pathway to more broad adaptive learning strategies.
> Spaced repetition is great for rote memorization of arbitrary facts
Traditionally, yes. But several people use the SR algorithm to simply do the same problems over and over (practice), without the memorization component.
Looking at a typical SR algorithm, you're not really going to memorize doing the same medium/high Leetcode problem unless you actively try to memorize it. The interval sequence will be something like: 2, 5, 13, 35, 120, etc. By repeatedly solving the problem so few times in such a long period (and while solving other problems at the same time): You're not really going to memorize it accidentally.
Bullshit. You can't get to mastery without memorizing (at least) the fundamentals into muscle memory.
Can you just "master" the 7 notes and what octaves are and all other theory to be able to play beethoven? Or should you play a few songs repeatedly till they're muscle memory practicing in parts or full and then moving over to other songs?
Should you learn that the integral means the area under the graph to "master" integration or do you still need to run through dozens of same/similar problems till you can solve them without an error?
You need to memorize but its a fools errand to start and stop at memorizing leetcode problems. You need to get the fundamental problems (binary search, bfs, dfs, sliding window) solve them repeatedly to a point it just flows naturally and then apply it to problems that build on them and still repeat those till they get natural too.
I believe this is incorrect, primarily because this approach has never worked for me. Incidentally, just today I found an article that has put this feeling into words really well:
I think that’s a bit missing the mark. The emphasis in the story, contrary to the parent comment, is that rote memorization and repeated problem solving, lead to understanding. Whereas most American education claims the exact opposite: that understanding comes before fluency.
A lot of my personal opinion follows, and I'd be very, very happy to hear dissenting opinions! I believe there are at least two valid goals for learning that are relevant to memorization:
1) Learning for mastery / competency to apply the skills to real life endeavors
2) Learning so you can meet some standard necessary to enable certain opportunities in life
It seems your opinion comes from a personal value bias towards #2 and I share that bias, but perhaps not as dominantly as you do (at the moment). For both of these, memorization can be very helpful, even optimal. But the application of spaced repetition would be different depending on the goal.
On memorization applied to learning for genuine personal mastery:
I believe memorization is optimal for foundational axiomatic knowledge. A good example is quick addition and times tables (and other arithmetic). I believe:
1) People should consider it a necessary life skill to be able to perform reasonably complex arithmetic manually in your head. For example, 4462+9241=13703 or 25120=3000. Many people will disagree on the nuances on where this begins and ends, and that's okay. For example some people will feel that people should be able to multiple 27123 in their head, others would feel it's okay for something like that to be done on paper. But for me, the point is that there exists a category like "things you should be able to do, but it's fine not to have memorized". More importantly, I believe it would be suboptimal to spend a significant amount of your life memorizing an increasing sample of these, but it's reasonable to spend time maintaining the skill of manual mental arithmetic and probably reasonable to spend time improving the speed and accuracy with which you do it.
-- 1a) A nuanced subset of this skill example is "estimation", for situations where it's appropriate. It might take longer for some people to manually calculate 27123 (or whatever) than for them to pull out their cell phone or ask their Apple Watch. If the situation only needs them to have an upper and lower bound within they might break it down to {lower_bound = 25120 = ((256)2)10 = (150210) = 3000; upper_bound = 30125 = 3055 = 150510/2 = 75010/2 = 7500/2 = 3750} and they could determine the answer is around 3350 or so, but absolutely 100% definitely between 3000 and 3750. And they could still do this type (if not degree) of estimation faster (and more stealthily, which can be important sometimes, like during negotiations/haggling) than pulling out a cellphone and calculating.
2) People absolutely cannot do arithmetic if they do not have certain foundational axioms memorized. If you do not have 1+1=2, 2+1=3, 3+1=4, etc memorized (counting), you literally cannot do arithmetic. That would be an absurdly low-functioning adult and essentially means you don't know how to count. But technically* you only need to be able to count to add, subtract, and multiply (by maintaining two counts simultaneously) any real numbers. Obviously just counting is a helpful, but not optimal, level of memorized knowledge to be competent at arithmetic. It would plainly be helpful to also memorize things like 7+6=13 and 152 = 30, and have instant recall for these rather than re-calculating them manually every time you need them. Which brings me to:
-- 2a) I believe it is helpful to memorize more than what many people think it is helpful to memorize. We generally are taught in primary school to memorize times tables up to 10x10. But perhaps it would be useful to extend this beyond 10x10, at minimum for some additional prime numbers which might show up frequently and cannot be quickly factored to smaller numbers. I could see it being super nice to have my times tables memorized up to 100x100, and additional addition/substraction memorized. I feel this way because despite having done well in three semesters of calculus and differential equations, I have always had a few "holes" in my brain's lookup table for certain simple addition and multiplication - specifically it really slows me down to need to manually calculate 86 (I have to do 832) or add 16+7 (I personally have to do 20+4+3). It's very frustrating to hit these basic roadblocks. Often I wonder how much happier I'd be if, at a young age, I had personally chosen to endeavor to memorize an unusually large amount of basic integer combinations, rather than just focus on memorizing the minimum necessary to get 100% on my K-5 coursework. Why do I not know 86? Because I took the less ambitious route of just breaking it down to 446 or 823 every time. Which brings me to:
On memorization for learning so that you can meet some standard necessary to enable certain opportunities in life:
1) I think we can all agree that some things are worth striving for, but are gated behind formal or informal tests which aren't relevant to the skills and temperament that actually enable people who do those things to do them well. For many people, that's "leetcode", but it can also be things like a college graduate memorizing and practicing ad nauseam a powerpoint delivery for a technical role for which they'll never formally present anything in their entire career. It could also be memorizing certain key motifs in a history or politics or English class which are actually wrong. The student may even know* its wrong, but it must still be memorized and regurgitated to get the best possible grade for their transcript.
1a) I think we can all agree that when cramming for an exam for which knowledge is not particularly desired to be retained, the best memorization technique is the one that takes the least amount of time to remember the facts just long enough to regurgitate them for the exam. I've discovered that this method is not the most commonly evangelized "spaced" repetition, but rather a Leitner system implemented in a way such that you actually minimize the time spacing as much as possible. For me, this uses digital or physical flash cards. Make a deck containing all facts you want to be able to maccess n days later during the exam. Go through the deck one at a time and any you get correctly, move to a new pile on the right; any you get wrong, move to a new pile on the left. Keep going repeating this with the left-most pile without breaks or pauses until you are getting enough % correct to get the desired score on the exam (with some safety margin), or you get exhausted or stop improving (then sleep and do it again the next day). This takes up far less actual time than spaced repetition, but you'll also forget the information fairly quickly as well and retain much less of it permanently.
--------------
To bring it back to the actual topic of memorizing leetcode: For me personally, I think memorizing some leetcode can be useful in three ways.
1) (For both mastery and credentialism) This is stretching the definition of leetcode a lot, but it's worth mentioning. I think it's a good idea to memorize syntax and functions/methods of the language, libraries, frameworks, and tools that you intend to use. I switch languages a lot at work, and I switch jobs a lot. It genuinely shocks me how much I can accomplish in some languages for which I can't even write a syntactically correct for-loop. Like if you put a blank paper and a pen in front of me, I could not write a for-loop -- but I can copy/paste one loop from language documentation or google result or ChatGPT and then copy/paste that line from my source code over and over until I have a working API or application or website from scratch in under a day. But I hate it, and not knowing all the syntax makes me slower. Not knowing what methods are available for a certain class is very frustrating. I don't need to have it memorized to do my job, but when I'm working in a language that I do know super-well, I have a much happier time doing the job.
2) (For mastery) Some leetcode could be considered "merely slightly more complex fundamental concepts". Nobody would argue that you should be able to write a for loop from memory in your language, but I think it's reasonable to say that it could very well be useful to be able to have instant recall of some amount of slightly more complex algorithms rather than needing to google them, like implementing sliding windows using trees. Or (for low-resource embedded programming) things like exponentially weighted moving average. Etc. What these are depends on what you desire to have mastery over, but memorization via spaced repetition can ensure that these "leetcode" techniques are always available to you the instant you need to implement them.
3) (For credentialism) Yes, it would be wonderful if we all had the brainpower to instantly figure out all leetcode problems fluently on the fly without awkward pauses. But most programmers can't, and it's better for the interviews if you can quickly, decisively, and confidently present everything that the interviewers are looking for. For this, I believe it can be genuinely worth spending some of your life memorizing leetcode if it means the difference between making $120,000 vs. $225,000 for the next five years. You probably won't ever use two-sum or flood-fill in your real job, but if it's the requirement to get that job, it can make a massive difference in your quality of life. Offloading brain-processing by memorizing leetcode also allows you to spend more of your higher-level thinking on connecting with your interviewers, figuring out the best presentation style tailored to them, and optimizing emotional expression during the leetcode portions of the interviews. Basically, leetcode isn't the only "hard" part of interviews, and it's useful to free up as much brainpower as possible available for the other challenging things that you need to handle simultaneously as well. Most of those other things don't have a "ceiling", any additional marginal efficacy might make the difference. I believe spaced repetition is best here instead of time-minimized Leitner method, because not only is it helpful to still have much of this memorized two years later when you might desire to interview again, but also because the complexity of the problems makes a time-minimized approach fairly infeasible.
On the topic of memorising arithmetic, I have found it surprisingly useful to memorise a few base ten logarithms. While most people have a good sense of numbers as an arithmetic progression, the geometric perspective is often more useful. (Things like how 3 is halfway from 1 to 10, but 10 is halfway from 1 to 100. (This is just the square root, of course, since that is a power of a half. But it's also useful to know that 2 is one-third of the way to 10, and 1.6 is a tenth of the way to 100. I.e. knowing some logarithms, you can mentally compute arbitrary fractional powers in your head.))
If you master the methods and ideas, you should be able to derive the answers on the spot. That's better than memorization because then you can actually deal with real interviewers who 1) Want to hear your train of thought and 2) Will give you random changes or variants to problems.
It's not about memorizing solutions, because in reality when are you ever going to reverse a linked list or balance a red-black tree? No, so if you're going to put in the effort anyway, you should learn the concepts and understand when to apply them so your knowledge is actually applicable. This will actually make you a better programmer, and you'll do better with the interviews that are actually good (I speak with experience giving interviews at FB).