Why not take notes? Whenever I read a book I want to remember, I just pencil a dash in the margin next to any key fact, insight, or quote. Then after I'm done with a few chapters I retype these sections into a mindmap. It probably only adds 10 - 20% extra time, but you're getting 1000% more value.
In general what matters isn't how much you read, but how much you retain and what sorts of connections with past and future insight and information. It's important to have the full experience of having realizations and making connections while you're reading, which is why I just make a dash in the margins as opposed to taking actual notes in real time, but I feel like by not circling back later you're cheating yourself out of the true value of learning.
Especially since you have no idea if the books you're reading are even true or not until you vet the facts with primary sources.
The problem with taking notes is that you'd have to do so for literally every book or article or video you ever watched, and then you'd have to refer to it every time you remembered any facts from them.
In general what matters isn't how much you read, but how much you retain and what sorts of connections with past and future insight and information. It's important to have the full experience of having realizations and making connections while you're reading, which is why I just make a dash in the margins as opposed to taking actual notes in real time, but I feel like by not circling back later you're cheating yourself out of the true value of learning.
Especially since you have no idea if the books you're reading are even true or not until you vet the facts with primary sources.