Doesn't asking that Caplan spend one second longer on his calculations automatically rule out everything else that anyone has ever said about the subject?
That's the most interesting part of this for me - nobody's saying "you're missing some important factors, and let's add them." People are saying, "I don't know that you included all the factors, so I am going to go back to reading opinion columnists for whom 'factor' is not a word."
Has everyone thought about the fact that the alternative to spreadsheets is policymakers putting their hands on their hips and saying, "Yep, $30k sounds about right for this month?"
> nobody's saying "you're missing some important factors, and let's add them.
Ok, so I’ll say it. Caplan doesn’t know what he doesn’t know. And neither did the people who he asked to check his spreadsheets.
What he is trying to do, in an informal hand-waving way, is something known as causal inference. There are formal, robust ways to do causal inference, which Caplan is not using. No amount of checking his arithmetic or adding more factors is going to fix that.
> Has everyone thought about the fact that the alternative to spreadsheets is policymakers putting their hands on their hips and saying, "Yep, $30k sounds about right for this month?"
Well, I can’t speak for everyone else, but I don’t think about that because what you claim as fact is actually a false dilemma. The actual alternative is to do good science using the correct methods.
The practice is not the “do good science using the correct methods”. The practice is “call for increased education spending for *waves hands* reasons”. Listen, this is the old Internet trick: people show up demanding rigorous models and evidence when they see something they don’t like, but what they like they demand nothing of.
Ha. It's a trick to point out that there are well-known, easy-to-understand, rigorous models to measure this kind of this, which the author has declined to use?
Why bother with spreadsheets at all then? If the author is going to do a bunch of arithmetic to convince us that some conclusion is correct, shouldn't we ask that he actually perform relevant and useful arithmetic?
1 + 3 = 4 See? I just proved that we should spend eleventy brazillion dollars on K12 education next year.
Okay, so is his claim that "nobody cared about my spreadsheets" is simply incorrect, because people cared very much about them, by pointing out that they are useless, because they are not doing causal inference ... or what's going on? :o
> but has anyone looked at the spreadsheet yet to see for themselves? I think we're proving him right so far, haha.
But we aren't.
The author has described his method in words. My contention is that the method he described does not pass the bar for good science. If my contention is correct, then it simply does not matter what is or is not in his spreadsheets.
Here's an analogy: If I told you that I determined the safest car on the road by measuring the distance to the moon, and that I had very very good spreadsheets for measuring the distance to the moon, would you really bother to check my spreadsheets?
I don't think you would. I think you would (correctly) say, "Hey, that's a really dumb way to figure out the safest car on the road and I'm not going to bother to check your math."
Now, I don't think that the author has done something quite that orthogonal to his stated goal, but I also don't think it's close enough that I want to bother checking his math.
That's ... not a fallacy. It's simply extraordinary claims require extraordinary evidence in action. (Eg. if your Bayesian prior is already edukashion gud, and someone says giv muniez to teacherz, you can say yes, ok, dollar. But if someone says no no, it's a waste of resources you can say okay the burden of proof is on you now, have fun.)
I'm not claiming it's a fallacy, my dude. I'm claiming it's a trick.
Since the prior is uninformed, it's obvious this isn't "extraordinary claims require extraordinary evidence" since both claims are extraordinary. It’s just that one is political desirous. Ain’t fooling nobody with the attempt at rational cover.
Caplan shouldn't be surprised at the reception of his work, because people have this prior.
Of course as you say high priests of rationality examine their own biases as they reach for the book, and by the time they flip it to read the recommendations on the back they are already empty vessels for unadulterated pure non-chill-filtered high-octane data and nothing else. But most people working in public policy are just lowly humans. :/
How much evidence you'd demand to change your mind depends on whether your prior was 50%, 90% or 99.999999999%. You shouldn't need more evidence to change back to 50/50 than it took to get you to your current beliefs - otherwise you'd have two equal cases and prefer the one you heard earlier for no reason other than that you heard it first.
Of course. I'm simply saying that Caplan shouldn know that people are entrenched in the pro-education position (because this is the norm for centuries, because nigh certainly everyone who is in the education policy sphere has gone through many years of the usual schooling and higher ed, and virtually all of these policy experts are pro-education).
Plus it seems despite his many spreadsheets his model is not able to do causal inference, which means his tables are very verbose narrations or appendices to his arguments in prose, and not a decisive proof.
> to change your mind
Unless one is in a state of 100% confirmation bias then ideally every little piece of evidence counts. Sure, we are not perfect walking-talking sentient infinite resolution mathematical distributions, so in practice we simply discard a lot incoming information, as we have meta-(meta?)-heuristics that we depend on. (Eg. if our friends and family and the news and even random blogs claim X we start to take it seriously, but still, if it doesn't really affect us our attention won't bother. And so on.)
And you know what? Just like with my hobbyist friend, I thought the spreadsheet they made were a valuable contribution! I thought it was BETTER than just guessing. Heck I know it was because so many mods in starsector bring in unintentionally broken equipment and make the game easy. Using their spreadsheet on those mods showed clear heuristics that indicated what in practice seemed to be true.
But my hobbyist friend lacked what I think is important here. The humility to see that the piles of multiplication for the handwavey guesses that they are. Caplan's spreadsheets, to me, made his argument more persuasive. But still not that persuasive. My priors already have education being a net benefit as highly probable, and I'll update, but not much.
Brian's schtick boils down to pointing out that sometimes, economists say things over and over again just because they, themselves, have heard them said over and over again - with the added trick (making it much more valuable) of proposing something better-researched than what they keep repeating. I guess that's my point, that this may be one of those. It'd be usual for him if it was.
That's the most interesting part of this for me - nobody's saying "you're missing some important factors, and let's add them." People are saying, "I don't know that you included all the factors, so I am going to go back to reading opinion columnists for whom 'factor' is not a word."
Has everyone thought about the fact that the alternative to spreadsheets is policymakers putting their hands on their hips and saying, "Yep, $30k sounds about right for this month?"