Claim: banging your head against the wall makes you smarter.
Experimenters: that sounds amazing! So we tried. We can report that all test subjects reported a massive headache. Unfortunately none seemed to get significantly smarter.
You: a negative result, well done! Time to celebrate!
And now I hope you will accept my apologies for exaggerating to the point of being rude, all in reply to your entirely reasonable question.
I guess my point is this. There has to be a cutoff where a claim is too obviously incorrect and testing is simply not useful any more. But the natural cutoff point depends enormously on perspective. Are you a car mechanic, a rally driver, or a complete layman? And as the latter, who do you trust?
My original comment was written to highlight the frustrating perspective of the car mechanic. But an interested layperson (or a non-expert handing out grant money) might have the more difficult time.
> I guess my point is this. There has to be a cutoff
> where a claim is too obviously incorrect and testing
> is simply not useful any more.
Such as claims of giant lizards ruling the Earth, or claims that all matter (and time!) was once a singularity, or claims that man had descended from - get this - apes?
Emphatically not. There is an important difference between 'outlandish' and 'in flagrant contradiction with billions of experimental results [0]'.
I am not saying that the difference is always easy to spot, but I think that you cannot deny that it exists.
[0] For a crude estimate, see 'number of collisions per second' at https://home.cern/resources/faqs/facts-and-figures-about-lhc . And those people are looking really hard for possible violations of the law of conservation of momentum: in this context it is called 'missing transverse energy' and it is a key method in the search for new physics.
But it seems to me that in the earlier EmDrive experiments, there were forces found that were not explained satisfactorily. Now, through better experimental controls, all of these have been accounted for. That sounds like valuable research to me.
Good point. It should both make future experiments more accurate and also debunk similar claims more easily.
Science isn't a bunch of scientists saying "we doubt it'll work"—science is a bunch of scientists experimentally proving it doesn't work.
The people funding the scientists decide what's worth testing. In this case, the possible benefit vastly outweighed the cost of testing.
To fix the head-to-wall metaphor: in some experiments, subjects who banged their heads against the wall reported being cured of all mental illness. However more rigorous tests showed this remarkable effect was actually a placebo caused by the subjects believing they were involved in making history.
Why not celebrate it?