This is a risk, yes. I don’t necessarily think it’s a big problem.
First off, you’re describing premeditated, wilfully fraudulent behaviour. I suspect a lot of the problems are either less deliberate (you react to new information by making changes to your hypothesis or protocol without stopping to think what that does to your experiment’s reliability), crimes of opportunity (“this is so close, there’s no harm in fudging it a little bit, right?”) or a combination of both. It’s ok to enact rules that fight pervasive small issues while doing nothing about rare malicious behaviour!
Even the malicious behaviour would become much harder though. First off, the strategy you described only works for setups where the data collection stage is relatively short. If it takes a long time, then the fraudster’s timelines start looking suspicious. Second, that type of behaviour could really harm you if caught.
First off, you’re describing premeditated, wilfully fraudulent behaviour. I suspect a lot of the problems are either less deliberate (you react to new information by making changes to your hypothesis or protocol without stopping to think what that does to your experiment’s reliability), crimes of opportunity (“this is so close, there’s no harm in fudging it a little bit, right?”) or a combination of both. It’s ok to enact rules that fight pervasive small issues while doing nothing about rare malicious behaviour!
Even the malicious behaviour would become much harder though. First off, the strategy you described only works for setups where the data collection stage is relatively short. If it takes a long time, then the fraudster’s timelines start looking suspicious. Second, that type of behaviour could really harm you if caught.