On the other hand, this is the sort of attitude that would make an informed skeptic reject the theoretical possibility for nuclear power, two weeks before the first nuclear bombs were used in war.
It took a team of physicists four years to disprove this result. Did they do it just to prove a point, or because they didn't understand the car analogy? No, professional scientists are rarely that callous with their time.
There was a slim possibility that our understanding of physics was wrong, and now we know for sure that this was not an instance of that.
But everything that's really interesting happens at the edges. No groundbreaking scientific result, ever, has happened by rejecting out of hand unlikely observations that violate established theory.
These weren't rejected out of hand - from the very first results there were ripe questions being asked (i.e., why did it take so long to measure thrust, and how were they controlling for thermal expansion, see even here on HN, quoted elsewhere in this thread - https://news.ycombinator.com/item?id=12962579 )
You're absolutely correct that building better testing to disprove it completely is a good thing to do, just to remove the extreme unlikelihood that one of the most fundamental laws in thermodynamics are incorrect, but extreme skepticism is definitely warranted when the claim is so huge, and people are able to find realistic flaws (sufficiently realistic as to be proven to be correct) that would explain it entirely.
I'm not arguing against the points you are making, but they are subtly different from what OP implied. It's hard to make specific claims against a vague analogy that just seems to embody a mindset, but that's what I'm trying to do. And it's not a strawman even if it would happen to miss OP's intent; you see this mindset everywhere.
My point was that OP appears to would have rejected this hypothesis out of hand, and that this approach that would be wrong if everyone that matters did it. Obviously there were people who mattered who did not, as the experiment demonstrates.
A posteriori it's difficult to put numbers on it, but I'd say this experiment was worth doing even if your heuristics put it at a 0.1% chance of succeeding. Obviously that's not the case for Mad Morty's 1000th garage-built perpetual motion machine, but multiple physicists obviously evaluated this hypothesis as less shitty than that.
All in all we would probably agree if we could just zero in on our key points, so I probably shouldn't have spent four paragraphs on this.
? A nuclear power plant was built before the bombs. It's main purpose was to generate bomb material, but they definitely showed the principle worked and powered a lightbulb with it.
I was making a rhetorical point, but it works if you switch out 'theoretically' with 'actually', and 'two weeks before' with 'while the first nuclear reactor was supercritical in Chicago'.
I'm pretty certain that the practicality of a nuclear chain reaction wasn't widely accepted in December of 1942, even though fission was first demonstrated by Meitner and Hahn three years before.
The bomb really wasn't known. Some people speculated that it may work, some people were sure that it couldn't.
I've finished https://www.goodreads.com/book/show/170428.Now_It_Can_Be_Tol... last week, and from what I understand, a lot of people didn't think that bomb is a possibility (for example, the Germans weren't really sure), and even when the Manhattan project was running several years, the scientists still weren't sure that the bomb will work as expected. There was a lot of variables in play, and for example, materials for the bombs weren't available for testing in large enough qualities.
One of the concerns, from what I understand as a layman, was that a critical amount of material will blow itself up too early, so that most of the material won't be part of the explosion. This was solved, I think, by some kind of neutron reflectors, or something*.
There are few chapters about how part of the Manhattan project was capturing scientists from France, Italy and Germany. When they captured them, their housing was bugged and there are transcripts from that time in the book. When the bombs were dropped, they generally couldn't believe that it really worked.
* One of the transcripts shows, that the Germans didn't think of this, and when they thought about the bomb, the thought about basically overloaded nuclear reactor, which would be big, heavy and impractical for use as a bomb.
It took a team of physicists four years to disprove this result. Did they do it just to prove a point, or because they didn't understand the car analogy? No, professional scientists are rarely that callous with their time.
There was a slim possibility that our understanding of physics was wrong, and now we know for sure that this was not an instance of that.
But everything that's really interesting happens at the edges. No groundbreaking scientific result, ever, has happened by rejecting out of hand unlikely observations that violate established theory.