How does the lack of a citation in a laymans article suggest much better research is needed on this issue?
You are arguing that "Perhaps that was a very unrepresentative group of gifted students and this result will never be replicated". Well perhaps not. Given people doing research are not as stupid as random people on the internet think they are, and that experimental results are a better source of knowledge than blind statements on forums, I would put very low priors on your assertion.
I went all the way back to your first post on HN the last time you flamed on this issue, and I get where you are coming from. But I don't think you get where I'm coming from. I simply think that everyone engaged in science, from the lofty Nobel Prize winner to the first-year graduate student grunt worker, has to be prepared to follow correct methodology, each day and every day.
It happens, among all subjects that are researched, that I am most familiar with the research on psychology of human abilities and education reform (especially the intersection of those subjects, education of gifted children). Most of what passes for "research" in those subjects is appallingly bad from a methodological point of view. I've seen very influential publications with recommendations on gifted education that consist entirely of "fudge"--made-up citations that can't be verified if you look at the cited journals, and "studies" with no description of how the data were observed at all.
When a press release writes "According to a 2003 study," I immediately look for a citation of the study. Scientific studies that get published all have citations. Serious scholars cite that which is published. My alarm bells went off when I didn't see any citation to any study to back up that extremely implausible point. If you want to talk about priors, my statement is that it is EXTREMELY implausible that "a mere 40 percent of gifted children will complete an undergraduate degree" by any reasonable definition of "gifted." There are severe economic limitations on college access for some learners (cited by me in another post in this thread). But my claim is that well over half of "gifted" learners in the United States in my generation or subsequent generations eventually complete a college degree, and that the "study" mentioned but not cited in the press release is bogus.
After edit: Whenever I think about the hard work that scientists do, I'm reminded of Richard Feynman's commencement speech "Cargo Cult Science"
"But this long history of learning how not to fool ourselves--of having utter scientific integrity--is, I'm sorry to say, something that we haven't specifically included in any particular course that I know of. We just hope you've caught on by osmosis.
"The first principle is that you must not fool yourself--and you are the easiest person to fool. So you have to be very careful about that."
Have you ever written to Peter Norvig, Google's director of research, with your comments on his article?
Perhaps he could reflect your point of view the next time he revises that article, if you think there is something there in need of correction. On my part, I post the article a lot because my late father (a chemistry major who extensively studied the philosophy of science) alerted me to the issue of replicating research findings when I was quite young. Odd findings may be the next great discovery. They may also be flawed, mistaken observations.
Honestly, read your post and try to apply basic scientific tenets to it. I see examples of bias, rejection of results without justification (peer reviewed support), misidentification of a report to the public as a scientific paper, a claim without any scientific support, and an assumption that researchers aren't trying to perform good science. When weighing up your anecdotal claims against real research, I would always accept the latter. Do you disagree?
Note: I have no problem with you at all, and I'm not trying to flame, but I do have a problem with posts promoting what I consider a misunderstanding of the process of science.
My general point is that you are claiming the research is bogus without even a good critique of the methodology, sourcing the original article or providing any evidence of your own to the contrary, and using Norvigs article to prop up your argument. Science doesn't work that way.
Norvig's material is great when applied properly, and I have no issue with it. To reiterate: I am not critiquing Norvig's article, I am critiquing your argument.
To reiterate: I am not critiquing Norvig's article, I am critiquing your argument.
To reiterate: any time I post the Norvig article here on HN, it is because I see a popular press report on something that appears to be a scientific finding, without any affirmative showing that proper science was done to reach that finding. As a reader in the general public, it is not my job to do unpaid scientific research to refute purported scientific findings, but rather my friendly suggestion to my fellow readers that they check whether a claim is put forth on the basis of verifiable research conducted by proper protocols. I presume that if a press release were issued by your lab for reading by the general public, it would link to a peer-reviewed professional journal where the research either has been published or is about to be published. Any time a Hacker News thread brings up a research finding, and someone says, "Here is the link to the published article," I upvote that, because we can all do with more experience here in reading more scientific journal articles about more subjects. (It's regrettable, by the way, that there are still so many journal articles behind paywalls. I currently have access to a JSTOR subscription; if I did not, I would have much less access to scientific literature online.)
It appears that you and I agree that scientific research should be conducted according to certain well established methods. You are certainly more learned than I in the specifics of statistics, as I have observed when lurking in other threads. But the thread in which you first caught my notice for decrying my frequent posting of the link to the Norvig article on reading research
caught my eye because I thought it reflected a rather odd misunderstanding of my reason for posting the link. I post not to dissuade readers from taking scientific findings seriously, but precisely the contrary, to persuade readers to distinguish genuine science from publicity-seeking by press release.
I think it is in your very first post on HN that you asked a question about corruption in certain unnamed places. I have lived in other countries, so it doesn't surprise me to find out that some government officials are more persuaded by bribes than by scientific research. More generally, even before I had lived overseas, I had studied enough history and enough psychology (besides the philosophy of science I mentioned above) to be aware that human beings respond to incentives to disregard truth for personal advantage. I am especially careful here on HN to encourage readers to check sources and facts before believing the latest purported scientific finding, because examples abound
of persons who make false claims (sometimes because of sincere self-delusion, and other times for more despicable reasons) that purport to be scientific claims.
If a claim comes along that is warranted by careful research, the research findings should be published somewhere--every science researcher has plenty of incentive to publish--and it would be enough, after I post the link to Peter Norvig's article, for someone else to post "Here's a link to the published study, for all of us to check." I like to learn more about science. I like for everyone to learn more about GENUINE science.
P.S. If we search all up and down this now quite lengthy thread, no one has offered even a scintilla of evidence for the proposition "a mere 40 percent of gifted children will complete an undergraduate degree." I am sure, noting that even most of the anecdotes of academic failure during high school in this thread relate going on to undergraduate study, that that statement is factually incorrect for the United States for any time since the Baby Boom. I attended high school with several bright learners who were turned off by high school and had poor grades in high school and who in several cases dropped out of high school. But all of my bright high school friends, without any exception, and even though they were not from wealthy backgrounds, gained undergraduate degrees within a few years of leaving high school. The burden of proof is on the unnamed "researcher" who made the implausible claim.
Jumping to conclusions about the quality of research when you have not seen said research is in my opinion a non-scientific approach incorporating personal bias.
A better statement you could make would be "I would like to see the methodology of this paper, as it's assumptions are very far from what I expect as a practitioner in the field". This statement shows us that your experience in the field leads you to a different personal conclusion, but you are not jumping to the conclusion that the research is false because of this experience.
And I just checked my universities news section and they don't provide links to the original research. Most of the time I assume the article is in press.
You are arguing that "Perhaps that was a very unrepresentative group of gifted students and this result will never be replicated". Well perhaps not. Given people doing research are not as stupid as random people on the internet think they are, and that experimental results are a better source of knowledge than blind statements on forums, I would put very low priors on your assertion.