>> But if you tweak the hypothesis just a little, the data suddenly confirm it
This is 'data mining' right? And I've occasionally wondered about this, since I don't work in a scientific field but did once make use of the scientific method for some research I did. And yes the findings weren't especially conclusive but I'm not sure I could've tweaked the hypothesis to make it work.
So, had I found something really interesting that didn't fit the hypothesis, is the 'right way' to conduct a new experiment from scratch? So say I did that, and used the 'tweaked' hypothesis, of course I'd find something interesting, because it's already there.
In this new 'pre-registration' framework, how can I correct the problem and pursue the interesting idea but keep the science in-tact? Because, if I used some sort of cross-validation at the outset and I have all the data available I presumably can't change the sample, so the hypothesis presumably has to change.
Refining an experiment is not wrong. What is wrong, like you say, is going on a fishing expedition until you find a result you like.
There are methods to account for follow-up experiments. Bonfaroni correction [1], for instance, requires you to increase your significance level with each new test.
It's harder than you might think to control for multiple comparisons. The Bonfaroni correction assumes that each experiment is independent, and so penalises correlated experiments unnecessarily harshly.
On the other hand, other tests typically require the researcher to make explicit assumptions on the correlation structure of the experiments despite the fact that it is not directly observable.
You are probably thinking of Sidak correction when you state independence is needed. Bonferroni correction does not need independence. You are absolutely right about Bonferroni being a severely conservative correction though -- at least the 'first order' one that uses only the first term of the Bonferroni inequality. One can take more terms to be less conservative but those aren't as easy to apply as you need to know the joint distributions over larger and larger tuples of events.
Another more recent technique for 'exploratory' yet correct technique is to exploit differential privacy and dithering.
That would be datamining done wrong. Its perfectly fine to look at data to provoke new hypothesis. But you should not be using the same data to confirm the hypothesis that it provoked. Either use fresh data or make sure that you still ensure correctness if you are reusing the data.
> The lifetime in the air of CO2, the most significant man-made greenhouse gas, is probably the most difficult to determine, because there are several processes that remove carbon dioxide from the atmosphere. Between 65% and 80% of CO2 released into the air dissolves into the ocean over a period of 20–200 years. The rest is removed by slower processes that take up to several hundreds of thousands of years, including chemical weathering and rock formation. This means that once in the atmosphere, carbon dioxide can continue to affect climate for thousands of years.
> Methane, by contrast, is mostly removed from the atmosphere by chemical reaction, persisting for about 12 years. Thus although methane is a potent greenhouse gas, its effect is relatively short-lived.
Even then, it's worse by weight over most timescales.
https://en.wikipedia.org/wiki/Global_warming_potential: "In the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, methane has a lifetime of 12.4 years and with climate-carbon feedbacks a global warming potential of 86 over 20 years and 34 over 100 years in response to emissions."
Yes, if you want to "permanently" (i.e. a thousand years at least) sequester carbon from biomass sources you have to burn them to charcoal and bury that.
You can also pump CO2 into deep coal seams, displacing CH4 which is then burned in a closed capture system. The CO2 released from burning the CH4 is also sequestered resulting in a net gain since CO2 preferentially adsorbs to coal at a roughly 3:1 ratio to CH4. Economically viable coal seams are widespread globally, and exist within 50 km of most coal burning power plants in the United States.
But doesn't the comment above yours suggest that the 'yellow' dot thing may no longer be an issue and that now there's variation in the dithering parameters. Suggesting extra ink is not required.
More annoying are the privacy concessions that are the result of secret anti-counterfeiting measures (which is what I assume the measures are for).
>> My question these days when I read such article, "What has the author done that is equivalent?"
Well I happen to (vaguely) know the author and have worked with him in the past and he has done a lot of great work, not all of it in the public domain though.
Comparing him with an organisation that employs 17k people is a little unfair but he comes out of it favourably IMHO!
Absolutely, but without stating the blindingly obvious the calorific content of some foods is so high that exercise can not be the dominant factor of that function.
This article [1] from last week seems to support that view. There was something else on HN on Friday that offered the same opinion but included a lot more studies. Can't find it now, but the message is the same.
That's a bit excessive. I did a ride of 3 hours on the weekend and put 1832kJ worth of energy into the pedals, that translates (at 25% efficiency) to 1750kcal total energy expended.
It's actually pretty difficult to eat back those 1750kcal if you don't go out of your way to stuff your face with sweets.
The point most people (and certainly the press) misses with the "exercise doesn't work!11" meme is that of course it will never work if you simply see it as some kind of tool for weight loss or necessary evil. You are just going to stop the moment you have your "goal weight". The point is to turn the exercise (rather, find a sport) into a habit, a hobby.
>> It's actually pretty difficult to eat back those 1750kcal if you don't go out of your way to stuff your face with sweets.
Disagree. Over the course of a week it only means over eating 250kcal per day. What's that an extra muffin? Seems entirely plausible to me. Of course if you're expending 1750kcal additional kcal a day on cycling then THAT's a bit excessive, IMHO.
This topic is explored in Michael Sandel's book Justice[1]. IIRC his conclusion is that we are products of our environment and as such we owe a debt to the environment that helped create us: university, school, parents and not least of all the society that helped to build everything that came before us.
So the question should not be how do you justify earning it but just how much can you justify keeping it?
Yep, I made the same realization as the OP in that changing things to any large extent required being part of the management. I got my break 10 years ago then 6 months after that I quit to go and work overseas to be with my wife & family.
Turns out being a programmer is a very portable skill, and good ones get paid well. Life isn't so bad.
I imagine if I'd spent the last 10 years in management I probably wouldn't have changed jobs and be just as frustrated, just in a different way.
But let's not get carried away here, the OP (and I) might have been pretty poor management material after all.
This seems pretty close to my experience. I remember thinking when I first encountered SQL Server Integration Services that this is 'visual programming'.
It didn't take very long to realise that as a solution SSIS was well suited to some simple tasks where the logical actions on screen matched those taking place on the database.
But as soon as those tasks became even slightly more complex, and this mirror was broken, the whole thing sort of fell apart. Then I was struggling to find ways to defeat the system to make it do what I wanted. It was with this realisation, i.e. that the solution was the problem, that I stopped using it.
But it's not just SSIS that suffered this way, ActiveBatch is another example and I'm sure there are plenty more.
> - it's not unknown for kids to have relatively poor reasoning and decision-making skills
Agreed. And one way to get better at decision making (if you're a kid) is having the opportunity to try and fail at it. So this isn't a good reason to track your kids after all, IMHO.
This is 'data mining' right? And I've occasionally wondered about this, since I don't work in a scientific field but did once make use of the scientific method for some research I did. And yes the findings weren't especially conclusive but I'm not sure I could've tweaked the hypothesis to make it work.
So, had I found something really interesting that didn't fit the hypothesis, is the 'right way' to conduct a new experiment from scratch? So say I did that, and used the 'tweaked' hypothesis, of course I'd find something interesting, because it's already there.
In this new 'pre-registration' framework, how can I correct the problem and pursue the interesting idea but keep the science in-tact? Because, if I used some sort of cross-validation at the outset and I have all the data available I presumably can't change the sample, so the hypothesis presumably has to change.