Physics is far ahead of other disciplines in this regard. Choosing your statistical test after you gather the data, selectively removing "outliers" after you gather the data, non-blind interpretation of pictures by humans who have a stake in the outcome and only publishing statistically significant results are all par for the course in e.g. neuroscience.
This paper [1] points out that commonly-used measures of statistical significance are downright meaningless when additional degrees of freedom are hidden in the way you describe.
It's even worse than that paper describes and this is something that every statistics 101 class worth its salt points out: if you are allowed to choose a statistical test after you've gathered your data you can prove any conclusion you want with arbitrarily high confidence. Note that the paper does not list choosing a test before you gather the data as a requirement. The only way to do meaningful statistics is the way splat describes: describe exactly how you're going to analyze it before you gather the data, then send the paper to a journal which decides whether to publish it before the data has been gathered, and then complete the paper by actually doing the experiment and adding the data to the paper.
