Regarding Mensa itself, or any claims to intellectual superiority: one of the big lessons of my life has been that being intelligent has no intrinsic value. If you are unable to manifest your intelligence in a way that can substantially (disproportionate to your effort) improve your life or the lives of others, then your mind doesn't matter. It has no impact in the world, and I am not the first person to observe that having a powerful mind is at many times a burden.
You can see that other people may be more stupid than you, forwarding simpler arguments and relying on less rigorous thinking than yours, but if they don't want to be convinced of that, then you are powerless to make them see it. They go off blissfully, and you burn with anguish. Again, if you cannot manifest your intelligence in a way that forwards your agenda, who cares?
To phrase it in a more confrontational way: if nobody is forced to contend with your mind, nobody knows it exists. Or to summarize it in a quote:
"[Intelligence] is like being a lady: if you have to tell people you are, you aren't."
-- Margaret Thatcher
This clicked for me during my final year of college, when a friend of mine was taking an LSAT preparation course. During a break, he was chatting with the instructor about realistic outcomes, and said earnestly that he hoped to get a score of 168 (which IIRC is a very high score.) The instructor, a man of maybe 30, who had been out of school for several years, scoffed and replied, "Good luck, I only got 165 and I'm a genius."
And what did the genius do when not teaching LSAT prep courses? Why, he worked at Target.
Edit: expanded a little on the first couple of paragraphs.
"And what did the genius do when not teaching LSAT prep courses? Why, he worked at Target."
Kafka was a clerk at an insurance company. Einstein was an assistant patent examiner. Wittgenstein was a gardener at a monastery and then a primary school teacher.
Van Gogh only ever sold one painting during his life, and that was to his brother.
There are hordes of other people widely regarded as "geniuses", who lived and died in dire poverty. Many others were cheated out of the fame and fortune they deserved by others who got the credit. And many more are probably still unrecognized.
The cold hard fact of the matter is that the world generally does not reward intelligence, "genius", or even hard work. True, some have the temperament and luck to scale the greasy pole of "success", but many others don't -- or choose to focus their energies elsewhere.
> Einstein was an assistant patent examiner. Wittgenstein was a gardener at a monastery and then a primary school teacher.
I take your point, but Einstein and Wittgentsein are not good examples. You list jobs they held, but those jobs don't reasonably express the success either man had in his lifetime. Einstein was also a member of Princeton's Institute for Advanced Studies from 1933 to 1955 (when he died)[1]. Wittgenstein was also awarded a PhD at Cambridge - without having done coursework or exams - on the basis of his Tractatus Logico-Philosophicus and was an idolized professor for many years there[2].
Neither man was by any stretch of the imagination an unrecognized genius - they were both absolutely recognized as huge intellects with superior achievements in their respective fields when they were alive.
My point was the OP appeared to be judging by the jobs an individual held at one point in time. Judged by the same criteria, Einstein, Wittgenstein, and Kafka would have fared no better.
The fact that Einstein and Wittgenstein held more prestigious positions at other points in their lives would not have spared them from an attitude like the OP demonstrated at the time they held the more menial jobs, and judging only by those menial jobs.
Who knows what heights of glory this particular Target employee will climb to in the future? History is even more full of people who once held menial jobs but then went on to achieve vast fame and fortune (and perhaps even be judged a "genius" by many).
We also don't know what this particular Target employee did before he worked at target and taught LSAT courses. Perhaps, like Wittgenstein, he'd been a lecturer at Cambridge and was awarded some prestigious degree.
Ok, I know that's stretching credibility. He probably wasn't. But my point is that we don't know his past or his future. All we know are two job titles and his boast of being a genius. Should we judge him based on just those couple of facts? Or should we withhold judgment? My vote is for the latter.
Finally, I never claimed that Einstein or Wittgenstein weren't recognized during their lifetimes, just that Kafka and Van Gogh weren't, and that many others who are generally considered "geniuses" now weren't either.
Your original post sounded very different from what you're saying now, but I apologize if I misunderstood you. Your last two paragraphs talk about whole lives and what the world recognizes without in any way distinguishing between the examples you gave initially. Kafka, Einstein and Wittgenstein are all listed in the same paragraph; again, there's no distinction between them there.
I'm not trying to nitpick, just to say that I doubt I was the only person who took the post differently than it sounds now like you meant it.
They were listed in the same paragraph because they were examples of people who are widely considered to be "geniuses", but who either held menial jobs at one point in their life, or who weren't "successful" (in the common sense of the term) at one point in their life (in Van Gogh's and Kafka's case it was at all during their life, in Einstein's and Wittgenstein's case, during significant portions of their lives, though Einstein was certainly vastly more successful than any of the others during his lifetime).
The OP was talking about one point in the Target employee's life, the point at which he was a Target employee and an LSAT teacher, and he seemed to be judging solely based on job titles. That's a very myopic, and I would contend, unfair vantage point from which to judge a person's life, contribution to humanity, or achievements... as the examples I mentioned (and many others that I didn't mention specifically) demonstrate.
Please look again at your original post. This whole business about "at one point in their life" simply isn't there. You didn't say those words. Just the opposite: in one of the later paragraphs, you talk about people "who lived and died in dire poverty". You seem to be moving the goalposts of the argument. Either that or your first post was very unclear.
> in Einstein's and Wittgenstein's case, during significant portions of their lives, though Einstein was certainly vastly more successful than any of the others during his lifetime
No, see, you're doing it again. Wittgenstein was regarded from early in his life as a genius. He was considered a very eccentric genius (that's putting it mildly), but in his own lifetime he was successful and well regarded. If anything, he held lower status jobs because he chose to run away from the world of fame and regard that he already had.[1]
You guys are both right. The point really is that prior to hitting a career grand slam (a Tractatus Logico-Philosophicus, a "On the Electrodynamics of Moving Bodies") one can find oneself in a menial job where one is looked upon as an unfulfilled genius. There's nothing to say that this Target employee isn't writing the next great American novel in his spare time.
In summary, the "menial" job in and of itself should not be seen as a failing for the genius.
Also most people even though hit a career grand never get recognized for it.
Gandhi never won the Nobel prize for peace, Just imagine- Gandhi!!! Although there is hardly anyone in the past century who did explicitly more for peace than he did. He not just preached, but demonstrated an entire moment for independence of India and even succeeded all on non violence.
Not that he complained about it. But the world never recognized it at the time.
I thought the point was that one can find oneself in a menial job where one is not looked upon as anything other than a bozo. Before the grand slam, nobody is going to recognize you as any kind of genius. Probably agreeing with you, but wasn't sure about the wording.
We all have plenty of examples of people who, though very smart, fared poorly in the world and then later, in their own lives or afterwards, fared much better and even became famous---because those people became famous.
The OP was judging a person by his job, and implying that the person in question couldn't be a genius because he worked at a menial job.
I simply provided some counterexamples.
Furthermore, white collar or not, I doubt that many would contend that Kafka or Einstein were some sorts of models of "success" if judged by their jobs, and few would judge them "geniuses" solely by looking at what they did for a living.
The fact that Wittgenstein gave away his inheritance is completely irrelevant. We were talking about jobs as marks of genius, not about inherited wealth.
However, I will grant that being born in to a wealthy family, being given a first-rate education, and being surrounded by highly accomplished individuals (as Wittgenstein was) does usually give the beneficiary a tremendous head start over most people.
I would certainly have bet on someone born in to Wittgenstein's highly privileged position making something of himself than on someone born in to poverty.
That said, plenty of people born in to tremendous wealth squander it and their lives. A couple of interesting, relatively recent documentaries on the subject are "Born Rich"[1] and "The One Percent"[2].
The OP was judging a person by his job, and implying that the person in question couldn't be a genius because he worked at a menial job.
No, I was judging him by his accomplishments -- a measure by which your counterexamples shine, and the LSAT instructor falls down. All of the people you mention were successes in some way outside of their job, and they met my other criteria: they disproportionately advanced the state of the world.
What made you think your LSAT instructor's accomplishments were limited to his job titles?
What do you know about this man apart from those two facts?
Kafka and Van Gogh were completely unknown and unappreciated during their lifetimes. Kafka even burned 90% of his work before he died.
There are countless other examples of people who were only discovered and appreciated after they died -- often long after they died.
Conversely, many (if not most) of those who achieved great fame and fortune during their lifetimes are largely forgotten now, or considered of minor importance.
"You were talking about [jobs as marks of genius], but nobody else was."
On the contrary, that seems to be exactly what you were doing. Your lack of knowledge or appreciation of someone's accomplishments and apparent eagerness to judge this person by the job they held is what I was responding to.
If you want to debate the merits of inferring accomplishment from a person's job, that's fine. To call that my main argument, and countering it with your sample size of three, is trolling.
Here is the point: if all you know about someone is that they run a cash register, that doesn't tell you that they are unable to do anything else. This argument does not rely on sample sizes.
No, here's the point: if you run a cash register, don't scoff at someone who aims higher than what you've achieved, and don't throw your genius status in their face when you do it.
If you want to argue that a person's job is not indicative of ther level of achievement in life, then you absolutely do have to argue inductively. But I personally do not care about that argument. It is a stupid premise, and gnosis is picking an argument when one is not being forwarded. And he is doing it in a completely fallacious manner (overgeneralization, tu quoque), against what I clearly stated was an anecdote and life lesson.
Who does that? A person's own experiences, related as such, are anecdotes, not arguments.
But still, don't you think it's a little premature to judge your Target worker? Einstein was a patent worker while he thought about and conceptualized Special Relativity. Do you know what that fella' was doing in his free time?
I still agree with the main point though - IQ doesn't mean anything except another avenue of high potential. Just like being extremely physically fit and a better athlete leaves you with high potential to succeed in something physical. A high IQ gives you the potential to succeed in something mental, but you still have to put in the work and do something with it.
You learn a lot more about Einstein's potential by testing his IQ than by looking at his job history. If it turns out that the Target Einstein is too lazy to do any physics, that's OK, the IQ test will still tell you vastly more about what he COULD do under the right incentives.
I don't think jessedhillon was saying that someone working a menial job must not be highly intelligent. His point was that such a person isn't getting much value out of being highly intelligent (he was presenting it as a supporting point in favor of his claim that 'being intelligent has no intrinsic value').
I believe that the idea that someone who works a "menial" job "isn't getting much value out of being highly intelligent" is the fundamental misconception here.
If you are a genius, your time is not necessarily well spent on ANY job, whether it is considered menial or not by the intellectually unwashed masses, unless it is for exercise, relaxation, sustenance, or pleasure.
'The second kind of social adaptation may be called the marginal strategy. These individuals were typically born into a lower socio-economic class, without gifted parents, gifted siblings, or gifted friends. Often they did not go to college at all, but instead went right to work immediately after high school, or even before. And although they may superficially appear to have made a good adjustment to their work and friends, neither work nor friends can completely engage their attention. They hunger for more intellectual challenge and more real companionship than their social environment can supply. So they resort to leading a double life. They compartmentalize their life into a public sphere and a private sphere. In public they go through the motions of fulfilling their social roles, whatever they are, but in private they pursue goals of their own. They are often omnivorous readers, and sometimes unusually expert amateurs in specialized subjects. The double life strategy might even be called the genius ploy, as many geniuses in history have worked at menial tasks in order to free themselves for more important work. Socrates, you will remember was a stone mason, Spinoza was a lens grinder, and even Jesus was a carpenter. The exceptionally gifted adult who works as a parking lot attendant while creating new mathematics has adopted an honored way of life and deserves respect for his courage, not criticism for failing to live up to his abilities. Those conformists who adopt the committed strategy may be pillars of their community and make the world go around, but historically, those with truly original minds have more often adopted the double life tactic. They are ones among the gifted who are most likely to make the world go forward.'
I think a far more revealing point about Kafka is his self-view and the fact that in his will he wanted all of his works burned. I don't think Kafka was ever a very happy man in his life, so you really got to wonder how much good did the "being a genius" do for HIM regardless of how much he contributed to the world and humanity and our culture.
I met a genius that worked in the shipping department of the first software company I worked for. (Back when people shipped software.)
But working in the shipping department was only what he did in order to persue his passion, which was bird watching and photography. That is where he expressed his genius, although he couldn't make a living from it, it made him happy.
There's a great deal of the population who spend their lives working in shops and so forth doing relatively menial work. Why they're doing it is not about intelligence at all, it's all about drive.
A driven person who's terrified of Maths and Sciences is unlikely to become a leading Physicist, but they might well power through enough basic sums to successfully establish businesses.
An intelligent person who's not driven might never achieve anything like fame, at least in their lifetime, but they still probably enjoy their free time and passions.
There are plenty of people doing "menial" work who have plenty of drive. The work is to earn a paycheck and their life isn't about that. The drive is centered on something that doesn't earn them a living (e.g. family, bird watching). Not everything in life is about work and money.
That's interesting. I'm an LSAT instructor. My first reaction to reading the instructor's comment was, 'Ha, only 165, he's no genius'.
But, MENSA accepts 163 as a cutoff LSAT score for entry. So if MENSA == genius, then he is a genius.
Among people who take the LSAT though, he's only 92nd percentile. He doesn't have much business teaching the test. Anyone who's at an Ivy League law school has a better score than him.
I completely agree that intelligence, on it's own, is near useless. It needs to be combined with the other elements that make someone effective. THEN, and only then, do you have the ingredients for something powerful.
Our society lionizes intelligence to the point that many people with little but intelligence smugly rest on their laurels, and accomplish less than their potential due to defects in social skills, initiative, etc.
On an hourly basis, you can earn more teaching the LSAT than as an Ivy League law grad. Mind you, I don't work 80 hour weeks, so I don't make as much on an absolute basis, but the work is more interesting, and I've branched out into LSAT books and an online LSAT course.
But mainly, I wanted to point out that the instructor is unreasonably arrogant. By comparison with his reference group, he's far from exceptional.
By implication, the Mensa cut-off is also far from 'genius'. Top 2% is 1 in 50.
To be fair, that's 1 in 50 from a group that was relatively accomplished in an academic setting before the test. But I agree that it doesn't qualify as "genius."
I may not have been clear, my posts mix LSAT percentiles and IQ percentiles.
The Mensa cut-off is top 2% of the general public. 98th percentile on an IQ test.
For the LSAT, they accept 163 as meeting that cut-off. Relative to other LSAT takers however, a 163 is 92nd percentile. So the top 8% of LSAT takers are qualified to enter mensa, or roughly 1 in 12.
I took the LSAT in 2009. I scored 166, which was in the 93rd percentile. Mensa lists their cut-off as the 95th percentile, so your 163/92 figure is inaccurate, at least for recent years.
Being an LSAT instructor is surprisingly remunerative. One of the big test prep companies pays $50/hour, which is about the same as what a first year associate makes at a big law firm (when you account for the fact the the associate probably puts in 3000 hours a year).
I'll add that being able to work well with others is usually what determines whether you can manifest your intelligence in a useful way.
Working well with others takes many forms: being able to inspire and lead are obvious ones; but also the ability to take constructive criticism from others and learn from their perspectives -- which can be hard to do when you perceive the other person to be less intelligent (and perceiving others to be less intelligent is a problem by itself).
I find interesting the backlash against those who may have any interest in knowing their IQ, or god-forbid even feel good about it. What your anecdotes tell me is not that IQ itself is meaningless, but that success has a large factor of luck involved.
It's a failing of our society that genuinely high IQ people are not out there working to improve the lot of mankind. This growing trend of derision for those who acknowledge their own raw intelligence is only making it worse.
If you think that "high IQ" and "genuinely intelligent" are the same thing, you are sadly mistaken.
Granted, IQ is correlated with intelligence. But there are lots of undeniably intelligent people who do not have remarkable IQs (eg Richard Feynman would not qualify for MENSA). Conversely your average MENSA chapter is full of people with undeniably high IQs and little sign of remarkable intelligence.
(Disclaimer. Based on my GRE scores, my IQ is about 1/10,000. Based on my experiences in the real world, I believe myself to be intelligent, but nowhere near what that hypothetical IQ score claims. I've always considered MENSA membership to be a filter for people whose only biggest accomplishment is a good score on an IQ test...)
The problem with these arguments is that we waffle on the definition of intelligence. Feynman was admittedly no genius. There's no reason to claim he was obviously highly intelligent (and therefore IQ is meaningless). He had many gifts, a .1% intellect wasn't one of them. This doesn't take away from his accomplishments nor the impact he's had on society. There's no reason to redefine intelligence just to make sure it includes guys like him at the high end of the spectrum. His work stands on its own.
But with that said, any definition that says that Feynman wasn't a genius is obviously wrong. In every environment that he was in, he was given the genius label because he obviously was one compared to (usually quite intelligent) peers. If a test says that he is not a genius, then the test isn't capturing what common usage says that genius means.
Conversely the flip side of the point stands as well. The fact that someone is labeled a genius by a test does NOT mean that they have whatever intellectual abilities would be required to better society. There is a correlation - they are more likely to have those abilities. But not a guarantee. And the fact that we see high IQ people failing to use their brains life is not always evidence that those people's gifts are being used. The true gifts may simply not be what was recorded on the test.
Perhaps Feynman was a poor example. I was going by his claim that his own IQ was 125 (or something to that effect). I suspect that there is more to that number and that we shouldn't take it at face value. I see no reason why he wouldn't score much higher.
You are so wedded to your theory that you are unable to accept facts at face value. Presented with evidence that Feynman's IQ was not exceptional, you conclude that he was not a genius. Presented with evidence that he was a genius, you conclude that there must be an external reason that he did not do better on the IQ test, and the test result can be thrown out.
The conclusion that you seem to think impossible is that someone can clearly be a genius and yet that genius would not be reflected on IQ tests.
Let me offer you two alternate hypotheses.
The first fits the criticism in the above post. Many questions on IQ tests are ambiguous - there are multiple logically possible answers, and to find the "right" one you have to figure out what line of reasoning the test maker was most likely to have followed. This is a question about conventions, and Feynman clearly was extremely unconventional. He therefore is an example of someone who was literally too creative to do well on the test.
A second possibility is that Feynman's gifts were overwhelmingly mathematical, not verbal. His graduate entrance exams stand as evidence of this, as do comments from people who have examined his work. See http://www.psychologytoday.com/blog/finding-the-next-einstei... for an example. Since an IQ test has a large verbal component, on which Feynman would have only been somewhat above average, he did not score exceptionally well.
I like the first theory better, but there is significantly more evidence for the second. But either way there is no evidence from anyone who knew him, including Feynman, that the IQ score he got is not what he should have been expected to get based on what an IQ test measures.
The fact is that there is overwhelming evidence that IQ tests highly correlate with other measures of intelligence (SATs, GREs, job advancement, etc). I'm sorry but one outlier does not invalidate the whole body of research regarding IQ tests. The reasonable conclusion regarding Feynman is to suspect other factors at play. I'm simply changing my opinion in the face of new evidence.
>A second possibility is that Feynman's gifts were overwhelmingly mathematical, not verbal.
And of course this would be the "other factors at play". There are plenty of people who are savant-like in specific areas but aren't a "genius" in the generic sense. In a mathematical/scientific setting, Feynman would appear as a genius. In other settings, he would not. There's nothing surprising about any of this that would warrant a redefinition of genius.
Where to begin? I'll just make random points in no particular order.
- When a man widely recognized as one of the top geniuses of his generation is determined by a test to not be a genius, it is not Feynman's status as a genius that should be questioned.
- "Correlated with" is very different from "the same as." See the second half of http://bentilly.blogspot.com/2010/02/what-is-intelligence.ht... for some sample calculations on exactly how meaningful the correlations measured between an IQ test and other measures of intelligence likely are. (Short summary. If there is a 0.7 correlation between IQ and "true intelligence", then people with an outstanding IQ should be expected, on average, to merely be of somewhat above average intelligence. And vice versa.) Thus the measured correlations are actually evidence that IQ tests are at best moderately effective at identifying true genius.
- The body of data that we have on IQ and intelligence is COMPLETELY CONSISTENT with the prediction that we should expect a top mathematical genius to have only a somewhat above average IQ. Feynman's relatively modest IQ is therefore not a surprise. The fact that it is not a surprise is again evidence that IQ tests are a flawed method of identifying genius.
- We actually have no real evidence that IQ, g, or other related measures are measuring anything directly meaningful at all. The arguments about this are complex, please see http://masi.cscs.lsa.umich.edu/~crshalizi/weblog/523.html for an explanation. As a side note, to the extent that by "genius" we mean something other than the weighted average of random abilities actually measured by IQ tests, we should expect that IQ tests are not that good at identifying geniuses.
Yes, I know that I just gave you a lot of complex stuff to digest. But please give me the courtesy of assuming that I have at least a passing familiarity with the subject of IQ tests, have put some thought into it, and my opinions should not be immediately disregarded without investigation.
At the very least I hope I gave you enough to cause you to reconsider whether "top 2% in IQ" effectively captures what the word "genius" means in common usage.
In Feynman's case it seems quite plausible that the 125 is not an accurate report of a rigorous testing environment.
He is certainly the type of fellow who would intentionally misreport his score for personal/philosophical/fun reasons, or not care enough to remember the number correctly decades later.
Why do people keep trying to argue that an IQ of 125 is not reasonable for someone like Feynman?
The 125 figure might or might not be exactly correct, but it is reasonable given other data we have about him. For instance, according to James Gleick's biography, Feynman was the only person to ever get a perfect score on the math and physics graduate entrance exams to Princeton. At the same time he scored below average on the verbal and history tests. (Note that the "average" at Princeton would be well above average in the general population.)
Feynman was an amazing genius. I have the deepest respect for him. That fact notwithstanding, his genius apparently did not evenly extend to all types of intellectual abilities. If so, it is unsurprising that he would merely score very highly on an IQ test.
> It's a failing of our society that genuinely high IQ people are not out there working to improve the lot of mankind.
Or maybe it's a failing of those high IQ people? The genius who proudly saves the codebase at Zynga every single day is completely useless compared to any single nurse working her shift right now. Even the doctor at the top of the hospital does not necessarily have a stellar IQ. What other way is there to put it? Should we feel bad for smart people because society tricks them into financial services rather than less-paid cancer research? (No one would look down on someone who does the latter.)
>Should we feel bad for smart people because society tricks them into financial services rather than less-paid cancer research?
No, we should feel ashamed that we've created a society where the brightest are funneled into financial services. We should feel utterly devastated that we let genius wallow in menial jobs. It is a failing of the system at every level that these people fall through the cracks and end up doing nothing with their gifts.
Raw intelligence is a resource that society should work hard to 'exploit'. Sure doctors and nurses save a lot of lives and that doesn't necessarily require genius. But compared to, say, the guy who discovered the double-helix structure of DNA, their impact on humanity doesn't even come close.
> It is a failing of the system at every level that these people fall through the cracks and end up doing nothing with their gifts.
Are we speaking of different groups of people? I agree that young geniuses should be treasured and given all the freedom they need to grow. But as adults, if they take on a job in HFT and stay there, it is not falling through the cracks anymore. They're irresponsible by choice, not because a faceless society forces them to (well, in most cases).
I agree about the double-helix discovery, but does anyone really need to do an IQ test to find out whether they'll be a scientist? I don't want to say "the proof is in the pudding", because there is lot of chance in science, but "the proof is in the job" comes close IMHO :)
I was attempting to comment on those on both sides of the spectrum, the finance gurus, and who fall through the cracks or otherwise don't thrive in our educational system. But I don't think laying the blame solely on those who choose a lucrative path is being completely honest with ourselves. People as a whole respond to incentives, and when we give a genius every incentive possible to buck science and research in favor of finance, we as a society bear a large amount of responsibility for this choice. We must change what we value as a society first, then we'll see this trend reverse itself. Admittedly, I haven't the slightest clue how to accomplish this.
It seems like a lot of people over think intelligence and success. Properly balanced or even an excess of serotonin and norepinephrine are what primarily ensures survival and procreation, which generally translates into happiness and sufficient wealth. Trying to measure and identify abstract cognitive ability like IQ tests do is almost somewhat of an autism spectrum-esque devotion. While some smart men and women may contribute greatly to society it's unusual for it to be a personal victory or indeed be as significant as dozens of better predictors of personal trajectory. Over-thinking things may even position excess cognitive function as a net negative drag on success.
I think the LSAT is a pretty good... well whatever it is.
It has the salient quality of being almost completely unambiguous. Out of 1000+ of LSAT questions I saw when studying for the test, I found maybe half a dozen that didn't have one unambiguously correct answer and four unambiguously incorrect answers.
Of course, it assumes a level of familiarity with formal logic and English grammar/vocabulary.
Yes, they're surprisingly good at it. I've taken and taught other standardized tests, and often find them frustrating or boring.
I once found an LSAT question I considered ambiguous. I sent in a complaint to the LSAC. They replied with a three page letter that convinced me I was incorrect.
We value lots of things in life that have no intrinsic value. Art, for example. There's nothing wrong with valuing intelligence too. It's what separates us from the apes after all.
That said, I don't disagree with your overall point. I've known many intelligent people woefully unprepared to deal with everyday life, and it always seemed sad to me. Two of them ended up as alcoholics, one with gambling debts. It doesn't matter how intelligent you are if it means you can't deal with life.
My best friend in HS happened to be a genius on paper. Got straight A's freshman and sophomore year, ended up doing less well junior and senior year, and, when he got to college, failed out. He now parks cars in a garage. I don't really care for IQ levels, I think they're mostly meaningless.
Thinking about this the other day, I was noticing that many people of great intelligence act as though it is a free pass to not putting forward much effort. Similarly they may neglect to pay attention to others who they regard as less intelligent than them (often their teachers or professors).
/"one of the big lessons of my life has been that being intelligent has no intrinsic value. If you are unable to manifest your intelligence in a way that can substantially (disproportionate to your effort) improve your life or the lives of others, then your mind doesn't matter."/
It saddens me the number of intellectually bright people I work with who don't grok basics like sound engineering principles. Your code is off little value if it's unmaintainable, or impossible to prove it works, regardless of how clever the implementation may be.
> if nobody is forced to contend with your mind, nobody knows it exists.
I'm sure this has already been explored somewhere, but I've always wondered what it says about us as humans that we consider our lives in isolation to be meaningless, but put a bunch of meaningless people together into a society, and impacting them becomes very meaningful. What's going on here, synergy?
IQ, "intellectual superiority" and improving-life-for-the-better or "happiness" are generally three separate things; or at least much more separate than we like to see them as clearly correlating if not causations.
And the people you meet at Mensa spread surprisingly many/all levels of society...
> But the biggest problem was that the idea of crossing the odd object out seems very strange to me in general. What is the odd object out in this list?
> Cow, hen, pig, sheep.
> The standard answer is supposed to be hen, as it is the only bird. But that is not the only possible correct answer. For example, pig is the only one whose meat is not kosher. And, look, sheep has five letters while the rest have three.
These types of questions irritate the theoretician in me as well, but to play devil's advocate a bit, are they really illegitimate questions? I would guess that your ability to correctly answer an "ambiguous" question like that measures your ability to effectively communicate with other humans, who regularly speak in ways that require the listener to resolve ambiguities that are at LEAST as severe as the farm animal question. If you think hard enough about it, sure, you can come up with a justification for any of the four answers (trivially, a cow is a cow and the other three choices are non-cows), but I would say that if you truly can't come up with the answer to that question that they "want" (barring any possible cultural reasons for not being able to do so), your intuition for pattern recognition could probably use some work.
This is not to defend IQ tests in general; I'm only arguing that ambiguous questions may measure something meaningful about your ability to learn or process information.
> I would say that if you truly can't come up with the answer to that question that they "want" (barring any possible cultural reasons for not being able to do so), your intuition for pattern recognition could probably use some work.
So it measures how good you are at blending in with other people, while being the entrance exam for a society of people claiming to be vastly different than other people. Hmm.
The ability to answer those kinds of ambiguous questions is extremely culturally linked. The whole idea about IQ tests is that they're supposed to be universal and culturally independent. The Cow/hen/pig/sheep question is clearly not culturally independent - what's important about those animals is a cultural question.
I don't think this question is a good example of being culturally linked. Rather it asks for knowledge about the world you live in. What characteristic of an animal do you think is most important?
- that it's koscher? (very cultural)
- the number of characters in its name?
- whether it is born out of an egg or womb (e.g. mammal or not)?
If you seriously can't figure this out I have to doubt your intelligence - however I wouldn't say that about all test examples.
Do you really believe that which characteristic is important about an animal is not a cultural question? If it seems "obvious" that's because your culture is invisible from the inside.
Sorry, but this is BS. First of all I'm European and my wife is Japanese. Asked her - same answer.
Second of all - just try some mind trick where you go outside yourself and try to observe something objectively, say as an 'Alien' who visits Earth for the first time. Which of the listed characteristics of a hen vs. a cow is important to you?
> I would guess that your ability to correctly answer an "ambiguous" question like that measures your ability to effectively communicate with other humans
So it measures how good you are at not thinking outside of the box? ;)
But I would say that "thinking inside of the box" is actually an incredibly important skill in practical situations. Probably 99+% of any functioning adult's thinking is "inside-the-box" (I realize that this is sort of a nebulous/meaningless statement, sorry). Things that we don't even think about, like looking both ways before crossing the street or watching our step as we get off the bus are all examples of inside-the-box thinking. But maybe a more interesting example of inside-the-box thinking (in the context of something you might want to actually test) is communicating effectively with other humans of a similar cultural background to your own.
So to get back to the portion of the article I quoted and maybe refine my point a bit, maybe it really isn't fair to say that any answer to questions like the farm animal one is more "natural" than another, but rather only seems more natural due to the biases of our own mental models that our evolution has optimized for survival rather than to achieve any measure of fundamental truth or purity. That's perfectly okay to me, as long as the test makes no claim to measure intelligence in any "fundamental" sense, but rather in a particular sense of intelligence that arises only as a human being living on earth. It's entirely a matter of opinion whether such a sense of intelligence is of any interest or relevance, though. Personally I think it is.
I agree. I am just not sure what task this is particularly useful for- from experience, it is really, really good for "feeling what an exam is probably about and acing it without studying much" :)
I would put it as measuring one's ability to determine the most salient category from the perspective of the person asking the question. In the case of the farm animal question, the context of the question implies the criteria that should be used to determine which is different.
I was raised in an abusive middle class household. My parents were not well educated. When I took the ACT test in 1975, neither my parents nor I even knew what the hell it was. I made a 23. I went to a unremarkable Tier 4 college because my friend was going there. I paid for it working as an "auger boy" on a gravel drilling rig for a concrete company for five years. I failed college algebra twice. I was put on academic suspension, and somehow talked my way back in. I made a C in algebra, a C in trig, and then took calculus from a handicapped polio survivor named Mr. Treese (not a PhD) who could barely walk or speak clearly. He had zero patience for bullshit but for some reason he liked me. After taking three classes from Mr Treese he suggested that I should enter the Math dept's annual calculus contest (I was a geology major). I actually won the contest (and a $60 check!). Afterward, I never made a grade lower than an A, and eventually moved on to earn an MS in CS. Today I think he was probably the single most influential person in my life.
Today we have two daughters who are both completing science PhD's funded with fellowships. I paid for their undergraduate education at Tier 1 schools but they have done the rest on their own without loans. You can bet your life that they knew what an SAT test was when they took it. All their lives we've told them that they were definitely very smart but that it just doesn't matter, that IQ is irrelevant. The only thing that matters is what you do, not how smart you are.
(I remembered this story because Mr Treese didn't care much for the idea that a "number series" problem has a "best" solution.)
This is a wonderful story. I think it highlights how important it is to have parents support you (mine would buy me books and take me to the library and entertain long, rambling thoughts in the car) and to have teachers and professors support you.
Thanks for your kind comment. I've thought of Mr. Treese often during my life. Here's an anecdote about him that's always cracked me up: One time he called on a student who wasn't, as usual, paying attention, asked him a math question, and the student gave a rude reply, "Your guess is as good as mine."
Mr. Treese fixed his eyes on him, gave him that crooked grin of his and said, "No that's incorrect. My guess would be much better than yours."
Years ago in our 20s my wife and I took an IQ test. I think she got in the 150s and I scored in the upper 140s.
But I felt I had done much better. When we looked at the scores, for several answers I had results that were clearly consistent with an internal model I could build of the problem, but it just wasn't the model the tester wanted.
So in a geometric progression, I noticed that each figure had an odd number of visible vortices. In a word analogy, I noticed that all of the words except one had an Old English root. And so on. Many times there were multiple patterns of resolution. How could one be better than another?
I could certainly understand her choices, I just thought they were no worse or better than mine. The fact that IQ tests are created by people with high IQs led me to believe that what is really being measured here is some sort of academic cultural cohesion; how much we begin to solve problems in similar ways?
Not necessarily a bad idea, but not what I think of when somebody says "intelligence test"
A properly _standardised_ test should at least have been administered to enough people that you were in the minority of the kind of people it was standardised against. Actually, a properly standardised test should have discarded questions like that if their answers weren’t reliable.
In any case, it is quite unlikely that the test you took could reliably distinguish between scores above 145.
Did any of the other word questions have answers based upon the linguistic properties of the words (rather than on the meanings of the words)? If not, then you evidently did fail at that portion of pattern recognition.
Be fair - for math patterns you're expected to choose the simplest fit. For language, the highest-order fit (e.g. meaning, not letter count). Why? Its cultural - thats how IQ tests are done.
That's true, but I don't see why it is a problem. You typically get example questions in these sorts of tests - it should be trivial to understand the types of fit that the test is requesting.
To be frank, I have very little time for people who claim to have overthought a test. Typically, they've thought enough to complicate the simple answer, but haven't thought enough to realise why the simple answer is correct.
Its not always possible to know which kind of question you're answering. What is the odd one in this set? Is that a math question, or language? How far along you are in the test can change the desired outcome; I'm certain the strategy being tested switches mode as the test advances to cover more capable subjects.
Short answer: you have to be familiar with the culture of testing to do well. Anecdote: I got 95+ on the GRE entering graduate school; I took advanced subject tests in subjects I had never studied. All it took was a knowledge of how the test worked.
IQ tests have long been criticized by a variety of groups. At the end of the day, it merely measures your ability to perform well on tests that are similar to an IQ test. Traditionally, this set was populated with academic tests, so the IQ could be used to predict your performance in schools (it was actually designed to predict the performance of French schoolchildren.)
However, as the school based subjects have broadened out from traditional liberal arts conceptualizations to include more serious discussion of alternative conceptualizations (CompSci being one example), the IQ test may be decreasing in efficacy of prediction even with regards to academic tests. I have no evidence for this, just an opinion. The issue you ran into seems to be a result of these differing conceptualization paradigms.
At the end of the day, it's an assessment of performance in a very limited subfield of life. I try not to take it too seriously.
That's not really true. IQ doesn't predict well on the high end, but it's useful for predicting the outcomes of middle intellect people. And it works better for groups than for individuals.
For instance, IQ scores are bad at predicting wealth but are the best predictor of adult poverty we have.
And the US military uses them extensively. They generally don't allow anyone in with an IQ score below 90.
Yes, every time the subject comes up, this whole crowd of relatively intelligent people go into a moral dance about how stupid IQ tests are.
They were never meant to predict genius. The point is to organize people efficiently when you're slotting them for guard duty or heavy equipment operation or officer candidate school. Someone who points out BS about non-Kosher pigs and obscure integer sequences is being clever for the sake of it. IE, "Look how smart I am."
And, as you say, the military considers about 100 million Americans dangerously too stupid for the army. They didn't do this out of arrogance.
This is constantly overlooked in debates over IQ, SAT, etc.
In most cases where these tests are used, it is far more mportant to separate the above median from below median, or +2 STDDEV from median, etc, and not to pick winners in the Krelboyne Super Bowl.
Even assuming there is a total ordering of intelligence, how would a single 300-question test rank 300 million people?
Why fill half the test with questions that 99% of takers will get wrong, just to differentiate among the top 1% of scorers?
Funny, the police in Police State USA prefer low IQ's
' on the theory that those who scored too high could get bored with police work and leave soon after undergoing costly training'
Or perhaps that being smart doesn't make you an enthusiastic state thug?
My dad made me take one when I was a kid and I did really well, but even then I thought it was stupid. The first issue is addressed in this blog post. The questions just aren't good enough. This is nobody's fault really, because it is very difficult to come up with a set of hard questions that measure your intellect and don't require lots of domain specific knowledge. It's either within a certain domain. If everyone taking the test was a Mathematician for example, the questions could be more clearly defined.
The second issue is that intelligence is multi-dimensional yet people try to measure it on a one dimensional scale. What's worse, the dimensions of intellect are somewhat elusive as well, so you can't just say something like IQ is the sum of values on each axis. Personally, I think that people that are high achievers in any field (business, arts, athletics, leadership, politics, science) are intelligent in some way. But when you look at all those people, there's no common denominator, no discernable pattern for intelligence.
Does it make more sense to say that intelligence is 'multi-dimensional' or to say that it's 'adimensional'? After all, IQ is a metric, not a direct measurement, as intelligence isn't itself directly observable. If we're defining 'intelligence' as being a quality of the human mind that's conducive to success at achieving one's goals, then it becomes something of a tautology to simply ascribe that quality to people who have observably achieved their goals.
If the IQ metric is intended to correlate success on certain types of tests with success in other contexts, then it may be reliable in certain regards - e.g. people with higher IQs do tend to hive higher incomes, as 3pt14159 pointed out below - but the actual nature of that common causality is still something that may not itself be understood sufficiently to describe it in terms of dimensionality.
I've always thought of intelligence as the ability to learn, which you could say is the maximum speed you can gain knowledge and understanding. A person may be able to learn in some areas very well, but not so well in others. Also, someone who has devoted much time to gaining knowledge and understanding may be much less intelligent as someone who could learn quickly but never puts their mind to it.
An analogy for this view of intelligence and knowledge is like a car's acceleration and speed. Some cars may be able to go very fast but they may take a while to accelerate.
As others have mentioned in this thread, what obviously matters more is how you apply intelligence, knowledge and understanding.
I picked your first example and googled it. The first result is a study demonstrating a positive correlation between divorce likelihood and IQ for people born in the 1940s in the Netherlands, a negative correlation between divorce likelihood and IQ for people born in the 1950s in the Netherlands, and a positive correlation again for people born in the USA in the 60s. The authors conclude that social factors (the difficulty of divorce and the negative social consequences) are important for this correlation.
It doesn't change that there's a correlation, but anybody using divorce rates as an argument for the effectiveness of IQ tests would have to be quite subtle.
I'm keen to know what the research is about cleverness and overthinking and multiple choice questions.
One of the most intelligent people I know IRL has trouble with the UK driving licence theory questions because she very rapidly provides 6 correct answers, and then has to try to detangle her correct answers from the choices.
I suspect that she'd do better if she tried when she was drunk (not the practical!!) because that would filter out the overthinking.
Of course, maybe this is all post rationalisation and some people just are lousy at some stuff.
It might not be that she's overthinking--it could just be that the test-writers don't have as much education/intelligence. I use ad-hominems sparingly, so let me offer a specific example for the beginner's license test in Halifax, NS, Canada:
In cold conditions, which of the following freeze before the others?
1. Bridges
2. Overpasses
3. Shaded Areas
4. All of the above
I wasn't entirely sure what a "shaded area" was, but I figured eventually that it must be some area under shade from the sun. Next, "all of the above" didn't make sense, because how can "all of the above" freeze before all of the above? I figured that water has a higher specific heat capacity than ground, and thus in cold conditions, the heat from the water would keep bridges warmer for slightly longer. I answered Overpasses.
Apparently the answer was "all of the above". The funny thing is, when I ask this question to people, I find a pretty consistent correlation between responses: for my non-university educated friends, they typically answer D. On the other hand, my university educated friends (whose majors vary significantly) tend to answer A, B, C, or "D doesn't make sense".
You would do well to assume the question isn't meant to be self-contradictory, and find a meaningful intepretation under this assumption. In this case, "before the others" could be extended to mean "before any other areas".
Actually the question should be rephrased to "Which of these can form precarious driving conditions in cold weather?" because that's what they actually want to know.
The problem as I see it is that the test-makers are trying to be overly clever and not ask the question they want the answer to. If I have to "extend" the language to say something it doesn't, the test maker has failed because the test question is the only source of truth.
This is the issue I see most often with these sorts of questions.
That rephrasing doesn't really work either. All roads can be precarious in cold weather; the point of the question is to determine whether or not you realise that some areas can become dangerous more quickly than others even though the rest of the road system has seemed "safe enough so far".
(And the "hidden question" is "did you know that shaded areas can freeze as quickly as a bridge or an overpass?" Most people who grow up around real seasons develop an intuitive understanding that thin, isolated objects cool off very quickly, but urban types are exposed to an artificial sense of ground temperature. The point is not so much to know D is correct before taking the test, but to realise that since both A and B are correct, then C must also be correct. It could have been better-phrased, yes, but a lot of questions on written driving tests are of that sort—they can teach as much as they test.)
* slight nitpick. I changed the basis of the question to encompass the need to reassess driving style in the presence of a change in weather conditions, as opposed to choosing specific instances of danger. That is not a rephrasing as I originally contended. My apologies.
If all roads can be precarious, the answer is all of the above, thus solving the dilema of the test taker understanding the need to be more perceptive in colder weather. We cannot control for all variables that would make a road slick or dangerous, and therefore cannot test them all individually, but must group them together into a situation (ex: road conditions changed).
If we accept that driving in and of itself is an active activity, then we should not have to distinguish between specific changes in road conditions for any change in road conditions should be enough for the driver to reevaluate their driving.
This question on the test should then be changed completely to reflect the fact that a change in road conditions necessitates a reevaluation of driving style.
I would further contend that "hidden questions", such as the one described, test one's ability to take multiple choice tests. If I know A and B, I have no need for C and there is no need for me to even look at it. In fact, if there were 40 options, "All of the Above" still has to be the correct answer, even if I have no clue what the other 37 options are.
Teaching is not the intent of a test. Unless, of course, we are assuming that the test taker did not learn anything before taking the test; In which case the intent is not testing them at all but ensuring that they have some basic skills to get by.
However I agree with your intent, the creation of questions for a test is not a trivial pursuit.
I think the "meta-context" must be taken into account: it's a driving license test, if I got it correctly. The examiner wants to make sure you'll be a careful driver. From this perspective, the mysterious D answer totally makes sense and is obviously what they expect.
I really have a hard time understanding this lack of "meta-intelligence" (or should we say "street smarts", that'd explain the success of at least some of your non university educated friends)
While it's evident that D is nonsensical, it's just as obvious that it is indeed the intended answer.
I expect if you are smart enough to detect it is nonsensical, you are also smart enough to deduce what you are expected to answer. I fail to understand why people nitpick themselves to death by a million cuts.
A bridge doesn't have to be anywhere near water. I would have answered "Shaded Areas" - why wouldn't a bridge or an overpass that's lit by the sun keep warm longer than an area that's in the shade?
I'm in this weird position of being excellent at the meta-reasoning required to ace a test and just smart enough to back it up, but not being very creative or powerful intellectually.
I was able to ace the National Latin Exam, my drivers' tests, and my handgun license test all without studying. Just from picking what seemed the most rational/logical in the context of a test.
I'd rather be an over-thinker. I'm terrible at a lot of things that demand pure intellectual horsepower. I'm primarily useful for just producing code, not really at higher-order thinking.
This person I mentioned - when she doesn't understand something she'll ask a question. And then she'll ask another question. And then she'll ask another question. And she'll keep going until she gets it. And because she's asking questions, and really thinking about it, she gets a deep understanding.
Me? I ask a question. Someone gives me an answer. I think to myself "ok, so I kind of get it" and then I drop it. So I have a broad but very shallow knowledge.
I have learnt to ask more questions, and to ask better questions. (Also, importantly, to use available online stuff to try to answer the questions I ask before I find IRL people.)
In theory, I believe there could be an objective way to determine the "simplest" answer if it is defined as having the lowest Kolmogorov complexity among valid answers (if I read the Wikipedia article correctly) [0]. In the context of an IQ test though, it is not a very practical solution.
It's actually an impossible solution, since Kolmogorov complexity is uncomputable. However, there are some related complexity measures which are computable, so that they are merely impractical rather than impossible to apply: http://www.scholarpedia.org/article/Algorithmic_complexity#O...
Of course, you can bet that the writers of the test did not use anything related to algorithmic complexity when deciding the correct answers to these questions. We can't show the exact complexity of the different hypotheses, but we can sometimes compare hypotheses. Imagine programs that generate answers according to the different hypotheses. Between the program that answers according to taxonomy, and the program that answers according to word length, which one requires a look-up table of animals and their classes?
1. Someone named "mistercow" is obviously biased on this question.
2. Another item to factor in, beyond lookup table size, is that questions are written in English, and the medium (spelled words) should not be snuck in as part of the content, except where sometimes it is done intentionally :)
Also, in classification problems like these, one should also consider not just how efficiently a solution chooses an answer from the set, but also how cleanly it isolates the cluster of items in question from the unnamed items. That is, since [cow,hen,pig,sheep] are all animals, more so than a random word is, animality should be part of the rule used to choose among them.
3. As blauwbilgorgel notes, Google's very successful solution to model picking is to slurp of everything published publicly online (and with Books, also many things published offline) and sample over the combined output of humanity. This is still biased towards written text, published text, and loquaciousness, but it's pretty good.
>Another item to factor in, beyond lookup table size, is that questions are written in English, and the medium (spelled words) should not be snuck in as part of the content, except where sometimes it is done intentionally :)
That just strengthens my point. The word-length hypothesis doesn't require any information about English. If we change our assumptions to say that the program is being fed the raw visual stimuli (as a human is), then the word-length hypothesis gets even stronger, since it merely involves comparing the widths of the stimuli.
But most of the information about the medium can be ignored when comparing hypotheses because it is constant across them.
>Of course, you can bet that the writers of the test did not use anything related to algorithmic complexity when deciding the correct answers to these questions.
Allow yourself to entertain the thought that they did. Imagine programs in our brain. Things that are easy to decode/(de)compress and require little energy/instruction are of low algorithmic complexity to them.
Start writing down farm animals in your mind. How long before you get to "hen"? You may have written down "chicken", but that would be too easy for a Mensa question, so you'd have to substitute that one :P
Wrong or obscure "answers" to a test question take more energy to produce, they are more complex, more random, less orderly. For me, "Cow" is the shortest program that gets executed first when I think of farm animals. It was likely one of the first programs for the test writer too.
Using Hackernews search API as a compressor, computing indexation distance between terms:
Cow + Pig 0.4843358722880948
Hen + Sheep 0.5328799348807779
Principles from Kolmogorov Complexity (Compression Distance) can give many approximate answers, even in the context of an IQ test.
Multiple choice is particularly easy, if you are allowed to note the result counts on queries like "Answer A + Keywords from Question" and "Answer B + Keywords from Question"
Speaking of cultural differences mentioned in the post -- You could even search locally on these terms and notice a difference in the semantic distances. If cows were to go extinct, in a 100 years, the search index would probably reflect on them being the least fit candidate for a set of three farm animals.
I'm not a mathematician, but I don't think that means what you think it does. It doesn't preclude being able to compute the K-complexity for some subset of strings, or for some fixed maximum complexity. Since the scenario here is checking which of a few relatively low-complexity options has the lowest complexity (eg generally could be expressed in English in <30 words), it seems it could be used in this context.
Many people like to argue for political reasons that IQ tests are meaningless or fundamentally flawed. One of the typical arguments, repeated in the article, is that the questions on a test require some cultural background and a smart person can fail them if they were raised in a different culture.
Following a similar line of reasoning, we could say that the vision test your optometrist administers requires the cultural context, after all if you were raised in a culture that uses cyrillic then you might not recognize some characters, and you would fail the test even if your vision was 20/20! Therefore, all vision tests are flawed, and the very concept of vision acuity is meaningless.
At this point in the conversation, I believe I'm supposed to accuse you of being oculonormative and suggest that you are blinded by your Visual Privilege. Would this be helpful? [y/N]
Another big downside to intelligence tests is they can make you think you are just that intelligent. (More commonly known as pride / arrogance. We all suffer from this, but I digress.) I've had conversations online with people who have scored extremely high on IQ tests. Now interestingly, this person assumed because of their high IQ scores that they were great at deducing patterns and reading people. They ended up coming to simply outrageous conclusions about myself that they were certain were correct no matter what I said.
So, I personally think IQ tests are worthless. I don't see any benefits that come from knowing what your IQ is. The only thing they give is arrogance, and as Albert Einstein (may have) said, "The only thing more Dangerous than Ignorance is Arrogance."
> > Cow, hen, pig, sheep.
> The standard answer is supposed to be hen, as it is the only bird.
I thought "It's the only one that goes around on two legs." And then: "It's the only one that reproduces via eggs." Maybe not the right answer for the "right" reason, but works...
But then, as I have gotten older, my IQ is probably half what it was when I was tested as a teen (not a candidate for Mensa, sorry) Probably because I'm more certain now that I don't know all the answers. :)
1) Hen - only one with wings / only bird / only non mammal / only egg-reproducer / only creature using 2 legs
2) Sheep - only one with more than 3 letters in name / renewable wool provider (eggs and milk are renewing?) Only plural without an S (sheep vs cows, hens, pigs)
3) Pig - only animal without renewing product (no milk, no eggs, no wool); only male name (cow hen and sheep are female);
I have always thought the point of this kind of questions is that as a candidate you are supposed to provide the right answer which you are expected to provide despite all the ambiguities.
This, I guess, should measure some kind of broader intelligence which overthinkers might lack.
I used to be in the Promethius Society, which is people with an IQ of 148+. I don't actually have an IQ of 148+ as far as I know, they just weren't smart enough to properly password protect thei Ning group. I have to say though after reading through the threads, most of them seemed vastly less intelligent than the smarter HN posters here.
The problem with Mensa tests, or any other generalized test that tries to identify patterns in humans, is that some people are taking themselves far too serious. IMHO Mensa is meant to give you an indication of not how smart you are but how perceptive you are in a particular kind of problem solving. And that’s about it.
Now if you score above 115 and then you expect the whole world to treat you like something special you miss the whole freaking point. I’ve seen people who because they score something like 130 or 140 are intentionally acting like whims or try to say something very profound and they end up speaking BS. They are very hard to work with and constantly expect to be treated differently.
Bottom line if you happen to be supersmart try to do something useful and meaningful for the rest of the society, otherwise nobody gives a flying fuck how smart you think you are.
> But it bugs me that I might not have been
> creative enough to fail their test.
I know Tanya, and she most certainly is creative enough to fail their tests. It's clear she was trying to conform, and did a job that was good enough for its purpose.
But she is certainly creative enough to fail, and way more intelligent (for pretty much any meaning of the term) than the test will have indicated.
It's for sure that those tests are not good for measuring the top end. The post calls in to question that any test could do so, but its clear that in any event, that test is not it.
I suspect it was far more valuable in pre-Internet non-population-dense America, where it was one of the few effective ways to connect with other smart people. It is extremely obsolete now. But "they" (whoever "they" is in practice) publish some nifty puzzle books.
At the time I'm writing this, yours may be the only answer in this thread that doesn't try to signal moral superiority to Mensa members by piously insulting them. (There are a couple of other comments that I would consider borderline -- not overtly insulting, but carefully suffused with cynicism.)
It is a nice way to meet other people that are (somewhat) smart (in a new city, or beyond ones social circle). What is especially good about it is the relative diversity of backgrounds that members are from (and different ages).
However the atmosphere at Mensa varies much from area to area (sometimes a bit pretentious, but in other areas very low key, pleasant and surprisingly open). And one should not take it too seriously.
That was pretty much my experience with the Mensa test too. The straight mathematics section was quite easy. All the pattern-matching ones were maddening because you could construct good chains of reasoning for all the options.
Great article. I experienced a similar frustration with the Stanford-Binet, and even more so in high school with the good ol' SAT (which Mensa also considers an admissable 'IQ' test).
It seems to me that a true "intelligence" test would also throw some generative problems at you: i.e., test you for as many correct answers to a pattern or sequence problem as you could come up with. I wouldn't advocate for generative problems to the absolute exclusion of deductive problems -- but the reverse is currently the case on most IQ tests, and it shouldn't be.
As for the broader point: there's been a scientific debate, for about as long as IQ tests have existed, over exactly what IQ tests are measuring, and exactly what IQ means. Most standard IQ tests are now regarded as measuring "g" ("general factor," sometimes called "general intelligence"). It may indeed be the most highly significant factor in intelligence, but even if it is (which, again, is debatable), it is just one factor of many. There are other factors, some of which are statistically significant. So simply isolating for the strongest factor is dumbing down the entire equation.
At any rate, "g" ends up being highly correlated with our concept of intelligence, but it is by no means a perfect or complete metric. And it does tend to favor a very specific type of intelligence, given that it is a narrow band on a broader spectrum. And vice versa: it excludes many whose minds we would probably consider genius caliber.
I don't think that "creative" people would fail the test. Creativity has nothing to do with finding smartass solutions to puzzles. It has little to do with even solving puzzles.
Creativity is seeing, connecting and coming up with things other people don't. The main prerequisite is usually grunt work. We only see the results, not the messy process behind James Dyson's vacuum cleaner with thousands of prototypes, or Bob Noyce's integrated circuit. What made him so creative was his knowledge and understanding of the world of semiconductors and the ability to question things.
I strongly believe that any inventor with a similar story would score high on an IQ tests and would not manifest his creativity by making up unlikely theories. Admittedly, its a sign of intelligence to understand the question and the context in which it is being asked. The world is full of opportunities to show ones creativity. Picking one of 4 answers in a test is not one of those.
I am a HN regular but I don't want to sound boastful so using a one time account.
I have a high but not super-high IQ, about 138. I went to a highly IQ-selective school, and as a result have met people who are smarter. Here is the interesting part: while they can do certain kinds of things better than me (puzzles, certain kinds of math, certain kinds of intricate code, analogies etc). Yet, I have gotten far more interesting things in life done than they have.
Even more interesting, I have discovered that I have thought harder and deeper about many problems than some of the exceptional IQ people I have met. As a result, I can simplify problems that they "only" (!) know how to solve. And this simplifying skill has proved to be extremely valuable in the real world. That skill doesn't seem to be directly implied by super-high IQ alone.
Great points, but "why join a club whose only entrance requirement is that you pass the test to join" is redundant. Every club with any requirement is exactly the same. Want to join a boat club and need the endorsement of one member? That's a test you have to pass to join, a social one. Want to join an old-fashioned gentleman's club? Then your test is 'be endorsed, well dressed, and have a penis'. It's still a test to get in. Want to join a motorcycle club? You're going to need a motorcycle and know how to ride it. Want to join Hufflepuff? There's a test for that, too.
Agreed it's redundant, but there's something else about it too. IQ is biological. You may just as well have a club for left-handed people, or people with green eyes. You'll have about as much in common with them.
Not entirely true, but the few Mensa meetings I went to were all 'social' ones (met at a restaurant), and I had nothing demonstrable in common with the people there other than having the same biological brain ability. We probably did have some things in common, but no one went out of their way to welcome me, and I didn't bother to try to make myself welcome either, so I quit going.
What I think we may have had in common is some of the same social experiences of being outsiders looking in to 'normal' societal groups. Certainly everyone's experiences are different, but often highly intelligent people will be ostracized (through their own fault or the fault of others or both), and being able to relate based on those shared experiences can be useful in some capacity. The group I went to didn't seem to be in to that.
sweeping statement, I understand. at core is my position as a materialist - I think everything is physical / material / biological. 'Intelligence' is simply an expression of that, and IQ as a measure of that is measuring something that's biological, like left-handedness or height.
My evidence... I've not seen any evidence to the contrary. We do see evidence that physically messing with brains causing changes in brain function. Brain damage can cause impairment to memory - short term memory is one of the elements of IQ that gets measured in tests that I've taken.
If you have evidence to the contrary that what we measure as IQ is somehow not rooted in biology, please share it.
I'm not here to prove the negative. Your opinion that nurture does not exist hand-in-hand is false.
"everything is physical / material / biological."
I don't believe in ghosts/souls/the immaterial, but this is irrelevant to social conditioning having an objective effect on IQ tests. Yes, the brain that stores information is "physical", but the information is not stored in birth.
One way intelligence manifests itself is through the ability to score well in an IQ test. Another way is through the ability to make an impact on the world. But I think another really strong characteristic of intelligent people is their tendency to lead a rich inner life. I get the strong impression most people go about their business without ever really thinking about how incredible it is that the universe not only exists but contains (at extremely low density) sentient beings capable of reasoning about it. Being one of those sentient beings is a remarkable stroke of luck. Intelligent people recognise that and celebrate accordingly.
IQ tests are just like anything else in life: If you practice enough, you will get good. If you are motivated enough to spend 10 hours a day practicing taking IQ tests, you will eventually get a high score. If you are motivated enough to practice the guitar 10 hours a day, you'll eventually become a guitar virtuoso. If you are motivated enough to spend 10 hours a day writing code, you'll eventually become a Rob Pike or Rich Hickey. If all you want in life is to be seen as smart by other people, then in my opinion, you need to re-examine some things in your life. I'd rather spend my time getting better at things that actually matter.
Ironically, this is similar to the original logic of the IQ test. If you are able to pick up and figure out the given questions quickly, you should be able to pick up and figure out many other things quickly, which is a pretty normal definition of intelligence.
There is no shortage of people who will do poorly at IQ tests because of cognitive difficulties and (sure) lack of motivation to practice. Does loading on factors like these make IQ tests less predictive of academic or career outcomes? No, both of those things also require a combination of cognitive ability and motivation.
Maybe, to some degree, that's true. But... I never spent any time learning how to take IQ tests, and have been tested at multiple stages - specifically IQ tests, and also the ACT/SAT.
First tests I remember, I was 9. Then ACT at... 15? Maybe it was 16. Then more IQ tests (3 different ones, IIRC) about 4 years ago. All showed me to be in the same X percentile, with a very small degree of variance between tests and over 30+ years of taking them.
And to repeat, I never spent any time learning how to 'take' a test. It was actually disappointing to get the test results and compare them to ~30 years earlier, and not see any change. I thought I'd have been 'smarter' as an adult, but I do realize it's not really testing that so much as... raw capacity perhaps? That hasn't demonstrably changed, despite all the life that's happened to me. No major health issues tho; I can certainly imagine that had I had a brain injury, my IQ test scores would be lower.
I have taken three and I have never applied for Mensa. Once in school when our biology class was curious about the topic; once online because somebody challenged me to it; once because it was in the TV magazine and I was bored, and I wanted to know if one really gets better at it. (I think I did)
I suspect you wouldn't have gotten demonstrably/statistically much better with repeated takings. Maybe if it was the same test multiple times, but even then I'm not sure.
A few years ago I had 3 IQ tests administered to me over several weeks. There wasn't any major difference between any of the scoring/ranking, despite the fact that I was taking multiple. I never got much 'better'.
I think it depends on many factors: Will I be told which answers were wrong and what would have been the right ones? I had similar problems with ambiguous questions as the OP and this might help me to understand what the test writers understand as "canonical" or "simplest" answer. Also, I am rather excited when I take a new type of test, so I guess my biggest jump would have been from the 1st to the 2nd test.
The standard answer is supposed to be hen, as it is the only bird.
I believe the standard answer is a "hen" because it's the only one that is gender-specific. In fact, it is distinctly the odd-one out because it doesn't even refer to a specific genus (or even family) of animal.
I'm all for creativity (I have the same complaint often) but there needs to be some sort of standardization in communication and society. A genius who can't communicate any of his/her ideas might as well be no genius at all.
Strangely, the first reaction I had was that a pig is the only thing that you need to kill to get something from. You can get milk, eggs, and wool from the other animals without killing them.
Not sure why I went in that direction, but I could imagine a lot of farmers and ranchers might think along similar lines.
Isn't that the point of the article? Everyone could pick one of those and reationalize why it should be the "standard" answer, but it doesn't mean it's the only right one. Good test would require candidate to pick an answer and then explain why, but that would make scoring such test too subjective.
Cow is specifically female, so it's not that. There are definitely a few different ways of cutting it, I went for sheep. It's a silly "test" of anything if it has a "correct" answer.
I've seen a few comments on HN and the blog about this question -- I think it's important to note that the article mentions this question was a set of pictures:
>>One test had many rows of small pictures, and I had to choose the odd one out in each row.
So the arguments about the number of letters, the order of the letter, etc wouldn't make sense here.
I've actually written an IQ test that a lot of people have taken. When researching how to put together an IQ test it became clear to me that from the beginning IQ testing was not, and never will be, a valid way to rate intelligence.
The simplest explanation is that "IQ score" is not fundamentally different than "SAT score" or "GMAT score" and is in a lot of ways worse because there are so many different interpretations and scoring methods for IQ. You can score 150 on one test but it might be scored completely differently than a test someone else takes. In other words, there is no agreed upon and copyrighted standard for IQ.
All of this type of testing really just sucks. All we really know is that IQ testing (or SATs, etc.) measures... something. And since people need to be quantified in some way, we use these scores for people-sorting. We TOTALLY overuse these scores without understanding them, and people feel great or feel horrible about their scores because of their usage (which is unavoidable) and for what it seems to say about them (which is very little).
It is hard to not feel good or bad about your score, even knowing all of this. We WANT to be rated, we want to score well and score better than others. That's human nature - just don't forget that your score will never impact what you can and can't achieve in life.
AFTER EDIT: Thanks to all who have replied for the interesting comments. I discovered this link while digesting replies I received on three different email lists to a request to name experts on mathematically precocious young people. (That was for work.) Tanya Khovanova, the author of the blog post submitted here, was one name suggested to me as an expert on precocious mathematics learners. When I saw her personal website,
posted to Hacker News (and other sites I read) before. I'll read more of her more purely mathematical blog posts over the next few days. I see one I can use right away in the local classes I teach to elementary-age learners.
On the substance of the post, I'm seeing several comments that equate "genius" to "person with a high IQ score." That was indeed the old-fashioned way that Lewis Terman (1877 to 1956) labeled a person with a high IQ score as he developed the Stanford-Binet IQ test. But as Terman gained more experience, especially with the subjects in his own longitudinal study of Americans identified in childhood by high IQ scores, he didn't equate high IQ to genius, and he became more aware of the shortcomings of IQ tests. Terman and his co-author Maude Merrill wrote in 1937,
"There are, however, certain characteristics of age scores with which the reader should be familiar. For one thing, it is necessary to bear in mind that the true mental age as we have used it refers to the mental age on a particular intelligence test. A subject's mental age in this sense may not coincide with the age score he would make in tests of musical ability, mechanical ability, social adjustment, etc. A subject has, strictly speaking, a number of mental ages; we are here concerned only with that which depends on the abilities tested by the new Stanford-Binet scales."
Terman, Lewis & Merrill, Maude (1937). Measuring Intelligence: A Guide to the Administration of the New Revised Stanford-Binet Tests of Intelligence. Boston: Houghton Mifflin. p. 25. That is why the later authors Kenneth Hopkins and Julian Stanley (founder of the Study of Exceptional Talent) suggested that is better to regard IQ tests as tests of "scholastic aptitude" rather than of intelligence. They wrote
"Most authorities feel that current intelligence tests are more aptly described as 'scholastic aptitude' tests because they are so highly related to academic performance, although current use suggests that the term intelligence test is going to be with us for some time. This reservation is based not on the opinion that intelligence tests do not reflect intelligence but on the belief that there are other kinds of intelligence that are not reflected in current tests; the term intelligence is too inclusive."
Hopkins, Kenneth D. & Stanley, Julian C. (1981). Educational and Psychological Measurement and Evaluation. Englewood Cliffs, NJ: Prentice Hall. p. 364.
So on the one hand there is the acknowledged issue among experts on IQ testing that IQ scores don't tell the whole story of a test subject's mental ability. A less well known issue is the degree to which error in estimation increases in IQ scores as IQ scores are found to be above the norming sample mean. Terman and Merrill wrote,
"The reader should not lose sight of the fact that a test with even a high reliability yields scores which have an appreciable probable error. The probable error in terms of mental age is of course larger with older than with young children because of the increasing spread of mental age as we go from younger to older groups. For this reason it has been customary to express the P.E. [probable error] of a Binet score in terms of I.Q., since the spread of Binet I.Q.'s is fairly constant from age to age. However, when our correlation arrays [between Form L and Form M] were plotted for separate age groups they were all discovered to be distinctly fan-shaped. Figure 3 is typical of the arrays at every age level.
"From Figure 3 [not shown here on HN, alas] it becomes clear that the probable error of an I.Q. score is not a constant amount, but a variable which increases as I.Q. increases. It has frequently been noted in the literature that gifted subjects show greater I.Q. fluctuation than do clinical cases with low I.Q.'s . . . . we now see that this trend is inherent in the I.Q. technique itself, and might have been predicted on logical grounds."
Terman, Lewis & Merrill, Maude (1937). Measuring Intelligence: A Guide to the Administration of the New Revised Stanford-Binet Tests of Intelligence. Boston: Houghton Mifflin. p. 44
Readers of this thread who would like to follow the current scientific literature on genius (as it is now defined by mainstream psychologists) may enjoy reading the works of Dean Keith Simonton,
the world's leading researcher on genius and its development. Readers curious about what IQ tests miss may enjoy reading the book What Intelligence Tests Miss: The Psychology of Rational Thought
by Keith R. Stanovich and some of Stanovich's other recent books.
Readers who would like to read a whole lot about current research on human intelligence and related issues can find a lot of curated reading suggestions at a Wikipedia user bibliography
occasionally used for the slow, pains-taking process of updating the many Wikipedia articles on related subjects (most of which are plagued by edit-warring and badly in need of more editing).
I also find it odd when people equate a high IQ with being a genius. In how I view the concept, one has to be a genius at something. I don't think there is a general "genius" category. In other words, I don't view it as a description of potential, but of unparalleled achieved mastery and accomplishment in something.
All an IQ tests, is your ability to do well on an IQ test.
It's not a particularly great measure of intelligence. The fact that you can often improve your IQ by 20 points (or more) just by practicing the types of questions that turn up should be evidence of this. Did you just become massively more intelligent relative to the population in the 2 days you spent practicing the questions?
Be fair; this is a correct comment, in line with the OP. Terman admited that IQ tests are only good at testing what is covered in the IQ test. A circular definition of IQ if there ever was one.
True creativity as well as true intelligence should INCLUDE THE ABILITY TO INTUITIVELY OR RATIONALLY "UNDERSTAND" WHAT IS THE SIMPLEST SOLUTION - meaning the simplest possible pattern that can be considered to consist of 3 or 6 or 9 shapes, one of which you have to choose in a standard figures based IQ test... "simple" might actually mean "being able to unambiguously explain it in the smallest number of words" for someone with a background in literature and philosophy and something similar for someone used more to the mathematical language... but in 90% of the cases there is an unambiguous "simplest" pattern that can be seen in the figures IQ test (not 100% because it's a statistical measure anyway, but still).
After all, don't we usually think of "smart" people as the ones that can take complicated things and make them simple or explain them in simpler terms (think Feynman, Einstein...).
...now about "cow, hen, pig, sheep" ...this is obviously a biased example by choosing 3 words of 3 letters and one of 5 and can be replaced with something that does not overlap the number of letters pattern on top of the number of legs or animal group one...
I'm no fan of Mensa, but there's one thing that I don't understand in this article - if the test was for non-English speakers and it contained pictures of animals, why was one of the possible answers that popped into the author's head "sheep - because it has five letters". That's language-specific, and should not be regarded as a valid answer in a language-neutral test. But the point is, it seems to me that the author is trying to find holes in the test. Of course that "hen" is not the only possible answer. As the author suggested, meat of pigs, for example, is not kosher. We can find things that differentiate any one of those animals from the rest. In any set, you can find some property of any object that will differentiate it from the rest. But that's not the point, is it? The point is to find the most obvious answer. The same goes for sequences of numbers. Yes, you can continue a sequence of numbers with any number that you like, and your answer will be perfectly valid. But the point is to find the most obvious one.
The English language is terribly ambiguous... even for native speakers. Unfortunately, test creators frequently do not appreciate this fact (a crowd sourced question review service for teachers and exam creators might be an interesting solution). One of my favorite examples is the number of alternate scenarios that this sentence could be describing:
the boy saw a mouse running in his pajamas
Counter-point: Throughout school, I often pondered whether the ambiguity in test questions was actually intentional. In the real world, you are often given problems to solve that are poorly defined, so perhaps this is good preparation? However, when working on real world problems, you can clearly state any assumptions you've made to arrive at your solution and this is not easily applied to answering questions on a bubble sheet test with very limited context.
Maybe you got a lot of those questions right, despite having reservations about them, and perhaps that's the whole point of the test. Despite there being confusing or ambiguous questions, maybe the test is designed to determine your ability to figure out what the test-makers designated as the correct response.
>On average they reject 98%. Mensa's cut off is the top 2% of IQs
These two statements are not congruent unless the distribution of IQs in people applying to Mensa is equal to the ditribution of IQs in the general population. I doubt that to be the case.
If I understand it correctly, they claim that you must be in what is thought to be the top 2%, in other words your IQ must be over 130-something.
But this does not mean that they reject 98% of the people that take the tests. People interested in taking that test must already be, I suppose, in the top 10% of the IQ distribution (125+?), or so they think. If this is true they they do reject 98% of the applicants but only 80%. Yet I suspect they reject even less.
Any source or third-party evaluation of their acceptance rate?
AFAIK, tests are prepared by some third party organizations (psychology departments?), scaled on representative sample of the population and are just bought by Mensa(and changed every couple of years). You should be aware that in different countries test with different SD (15, 16 and 25) are used. I heard from one guy who was responsible for testing in my country that about 25% of people taking the test get accepted
I joined Mensa and left after three meetings. It seemed to me that they mistook being able to think quickly for being able to think clearly, much the same as a teenager who thinks he knows how to drive fast just because he has a car with a powerful engine.
It would be amusing to have a bunch of people take the same IQ test, then tally all the plausible yet contradictory answers they produced to the same questions. Done well, that could be an effective demolition of the whole charade (though not in the sense that it would get anyone to change their mind about it).
The trouble with such tests is that they like you to be intelligent, but not too intelligent. So what they measure is efficiency of conformance. Is that what we should be optimizing for? I don't think so.
It seems absurd that something so obviously unintelligent would be the key to measuring intelligence.
Edit: oh, I see I'm just repeating what the OP has already brilliantly said. What a great and devastating little article.
Entry test for MENSA I had about 5 years ago didn't have any words or shapes of any real-world objects. There was only one kind of questions. You had to pick image that would fit in the right bottom corner of (usually) 3x3 image grid. It was all about picking apart those objects and noticing how their elements change as you move from row to row and from column to column. Me being programmer given me some advantage I think because I perceived some transitions as bitwise operations and frame by frame animations.
Having given my share of IQ tests, personally I always felt that they lacked in many sense. Once while researching about it I came across theory of multiple intelligence by Howard Gardner (http://en.wikipedia.org/wiki/Theory_of_multiple_intelligence...). Though not completely clear the basic idea behind it seems more logical to me.
A true genius doesn't want fame, fortune, awards, he doesn't want to stretch hands, hell, he doesn't even want to sign autographs. All he want is peace, out of the spotlight, in absolute and maddening silence, to keep doing what he does best.
Never quiet thought of it like this. There some things that would seem obvious to me but reading this realise perhaps not so much. Anyway IQ tests have long fallen out of fashion as they only measure a certain type of intelligence
Fashionable or not, intelligence tests are still much more indicative of (e.g. academic) potential than many other things people use to make hiring decisions.
IQ is worthless in a vacuum -- much like a computer. Though they have magnitudes greater computational power than the smartest human, without humans, they have no purpose.
Apologies, apparently Internet irony once again flew by undetected, my post was to highlight that to be in the top 0.1% is hardly that rare or unusual
As Tim minchin says, even if you're 1 in a million, there's still 7,000 of you
IQ tests share this property of undervaluing creativity and overvaluing conformity with academic work in general.
Witness my favorite renegade intellectual, Robert Pirsig, from his Wikipedia page:
'While doing laboratory work in biochemistry, Pirsig became greatly troubled by the existence of more than one workable hypothesis to explain a given phenomenon, and, indeed, that the number of hypotheses appeared unlimited. He could not find any way to reduce the number of hypotheses--he became perplexed by the role and source of hypothesis generation within scientific practice. This led him to an awareness of a (to him) previously unarticulated limitation of science, which was something of a revelation to him. The question distracted him to the extent that he lost interest in his studies and failed to maintain good grades; he was finally expelled from the university.'
I score pretty well on IQ tests, but only by suppressing a constant incipient rage at many of the questions, and a constant effort to ask myself "what would be the most boring and conformist interpretation of this?" That is usually the "right" answer.
Witness my favorite renegade intellectual, Robert Pirsig
I went to high school with a classmate who used to have the Pirsig family over for dinner fairly regularly. Robert Pirsig's father Maynard Pirsig was a law professor at the University of Minnesota, where the classmate's father (and, years later, I) studied for a law degree. I think, having read only Zen and the Art of Motorcycle Maintenance among Robert Pirsig's writings, and after limited personal acquaintance with Maynard Pirsig, that Robert Pirsig takes a lawyer's "that's debatable" attitude as he approaches scientific and philosophical questions, which bogs down his quest for certainty.
AFTER EDIT: I've seen an interesting pattern of drive-by voting on this comment, but I'd like to know more about what other people who are acquainted with the writings of Maynard Pirsig (father) and Robert Pirsig (son) think about the influence of the one writer on the other.
I'm not familiar with the Pirsigs, but I certainly do not consider the attitude in question to be lawyerlike. While lawyers do occasionally face philosophical conundrums, they also tend to have the desired answer known to them. This is considerably easier than to, say, set a prior for the domain of a model parameter, where your preferences are in a vague war between convenience and justice. The internal debate arises when working from first principles and finding too few.
Interesting. I first read ZAMM first when I was 15, and really enjoyed it, but I would strongly encourage everyone who has not read "Lila" to please read it, as Pirsig basically retracts most of what he said in ZAMM. In his own words, "Lila" is his "smart child", and the book he wants to be remembered by. It is a much better book than ZAMM.
If anyone is interested, I just discovered William James Sidis's "The Animate And The Inanimate" here:
Science has several rules of thumb, Occams Razor says that the simpliest is usually the more likely, and the assumption that the universe operates according to universal, consistant mathematical rules means that you should favour theories that match every other part of observable reality, the rule of thumb that extraordinary claims require extraordinary evidence is like occam's razor and means, again, choose the simpliest explaination that matches your evidence.
Hence, you can sort theories, and then try to devise tests to falsify each one to differenciate between them.
The problem with Occam's Razor is that it requires you to identify the simplest explanation. In fields like for example molecular biology deciding what is 'the simplest' often is difficult and debatable.
The author didn't name Occam's Razor, but she did touch on the idea that simplicity can be an unhelpful or ambiguous criteria to apply:
"Usually when you are asked to continue a pattern the assumption is that you are supposed to choose the simplest way. But sometimes it is difficult to decide what the testers think the simplest way is. Can you replace the question mark with a number in the following sequence: 31, ?, 31, 30, 31, 30, 31, … You might say that the answer is 30 as the numbers alternate; or, you might say that the answer is 28 as these are the days of the month."
This isn't to say that Occam's Razor isn't helpful -- it often is, if for no other reason than the fact that model/theory with fewer "moving parts" is simply easier to work with than another with more (and a similar level of explanatory power). Just that simplicity sometimes can't help or that it can be a value judgment.
I think in this case simplicity does reveal the answer. I would put 30 because the sequence becomes a repeating sequences of 31, 30. Answering 28 requires making the assumption that the sequence of numbers represent days of the month, which has a higher information entropy making the sequence less simple.
In a lot of modern biology however you will have tens or hundreds of thousands of equally simple competing hypotheses. Biologist have had to adapt to this by using multiple hypothesis testing methods, but Prisig was working well before this became standard, so perhaps this was the barrier he was running up against.
Pirsig actually came up with an entire metaphysical system in response to this. It's in his book "Lila". Basically it deals with what "good" (as in a good theory, or answer) is, and the different kinds of goodness or Quality, as he calls it, there are.
It's quite detailed, but his first distinction is between 'static' and 'dynamic' good. A test-writer values static good more highly than dynamic, and while this is justifiable in the circumstances, in the big picture, dynamic good is the real deal.
I think that Occam's razor says, that if we have several theories that describe reality with the same accuracy, we should chose the most succint (least redundant).
I don't think of it in terms of redundancy, but of introduced entities. That is, if two theories have equal explanatory power, but one theory requires ten new concepts, but the other one requires one new concept, prefer the latter. This interpretation is, I think, the spirit of what he intended: http://en.wikipedia.org/wiki/Occams_razor#Ockham
That's a good way to put it. If you want to be mathematically rigorous about it, you could phrase it as a statement about probabilities: for any A and B, P(A and B) <= P(A).
And if more than one theory is more-or-less equally probable, simple, and in accord with observed reality then you test all of them to see which you can eliminate. Repeat.
Yes. The context that has to be taken into account includes the social context, ie. 'I am sitting in a room taking a test, I cannot request clarification of the question, the person who wrote this test may be far less intelligent than me, etc'.
By making an effort to choose the possible answer that I perceive as the most boring and conformist, rather than one I might find brilliant, funny, novel, and suggestive of further even more interesting ideas, I am pursuing a strategy of "favouring theories that match every other part of observable reality", as you put it.
Is there any reason that in this case and in the case of the parent comment, there's a tendency to jump to conclusions about the intelligence level of the test/puzzle writer? Do truly intelligent people never write lousy questions?
When I was young, I was furious at "Skip the first letter, keep every other letter, and what does it spell?" Data was something like "ghfeklmlto", and my answer was "hfeklmlto", of course.
Bayesian uses a broader prior -- a distribution over possible prior distributions of facts, instead of assuming a single possible prior (Gaussian). In that sense you could sort of squint and say that that's the contrast.
Technically, you're allowed to use whatever prior you like. A beautiful thing about Bayesian inference is that it tells you how to update your prior, without making any assumptions about what it may be.
Frequentist approaches do not use any prior at all. Indeed some of the issues that frequentists wind up getting genuinely concerned about, such as repeated significance testing errors, are clear logical fallacies if you're doing any sort of Bayesian analysis.
Not sure what you mean by 'frequentist', but one thing is for sure: the more hypotheses that occur to you, the longer it takes to go through them testing each one not only for consistency, plausibility etc, but also for how likely it is that you think the test-writer would prefer the rationale for that answer. If only a few hypotheses occur to you, and they tend to be in the same ballpark as those that would occur to the average test-writer, you have an advantage.
My understanding is that the approach of coming up with a hypothesis and testing it is called "frequentist inference", in opposition to "Bayesian inference". However, I'm not terribly well versed in the subject and was hoping somebody could give a clear explanation of the difference between the two.
"Bayesian" and "frequentist" are descriptions of interpretations of probability more than of methods of inference.
Briefly and inexactly: A frequentist says that a probability is the answer to a question of the form "if we repeat this situation many times, what fraction of the time will this happen?"; a Bayesian says that many other things, such as (idealized) subjective degrees of belief, behave like probabilities -- i.e., they obey the same mathematical rules -- and that anything that obeys those rules deserves to be called a probability.
There is such a thing as Bayesian inference, but its opposite isn't "frequentist inference" but something like "classical hypothesis testing". If you take the frequentist view of probability then you will reject questions like "how likely is it that this treatment works?" because either it works or it doesn't -- there's nothing for a probability to be a long-run frequency of. But of course that really is the kind of thing you want to know, so you'll look for other similar questions that do make sense, such as "If the treatment doesn't work, how likely is it that we'd get results as impressive as these?". Asking that question leads to "Neyman-Pearson hypothesis testing": form a "null hypothesis" (the treatment doesn't work; the two groups of people are equally intelligent; the roulette table is not rigged by the casino; ...), do some measurements somehow, figure out how likely it is if the null hypothesis is right that you'd get results as unfavourable to the null hypothesis as you actually did, and if it's very unlikely then you say "aha, the null hypothesis can be rejected". Here, "very unlikely" might mean probability less than 5%, or less than 1%, or whatever.
If you've ever seen an academic paper in science or economics or whatever that says things like "eating more white bread is associated (p<0.01) with being a Mahayana Buddhist", that "p<0.01" thing is that same "probability of results as unfavourable to the null hypothesis"; in this (made-up, of course) case it would be something like "probability of the chi-squared statistic being as large as we found it, if bread consumption and Mahayana Buddhism were independent".
Bayesians, on the other hand, are perfectly happy talking more directly about the probability that Mahayana Buddhists eat more white bread. However, then another difficulty arises. Obviously the experimental results on their own don't tell you that probability. (Simpler example: you know that a coin was flipped five times and came up heads every time. How likely is it that it's a cheaty coin with two heads? You'll answer that quite differently if (a) you just pulled the coin out of your own pocket or (b) some dubious character approached you in a bar and invited you to bet on his coin-flipping.) The probability after your experimental results are in is determined by two things: those results, and what you believed beforehand.
On the other hand, here's a possible advantage of the Bayesian approach: Instead of just saying how confidently you reject (if you do) the null hypothesis, you can talk about the probabilities of various different extents to which it could be violated. (Imagine two medical treatments. One is more confidently known to have some effect than the other -- but the effect the second one might have is ten times bigger. You might prefer the second treatment even though the risk that it doesn't really work is bigger.)
So the Bayesian statistician might end up saying something like this: Here's a reasonable "prior distribution" -- i.e., a reasonable assignment of probabilities to the various possibilities we're interested in, before the experimental results. Now here's what our probabilities turn into if we start there and then take account of the experimental results.
Or they might just describe the impact of the experimental results, and leave it up to individual readers to combine that with their own prior probabilities.
Here's the recipe that's at the heart of Bayesian inference: Take the prior probability for each possibility. Multiply it by the probability of getting exactly the observed results, if that possibility is the case. The result is the final probability except that the resulting probabilities won't generally add up to 1, so you have to rescale them all by whatever factor it takes to make them add up to 1.
A similar thing occurred to me not too long ago, when I was learning about type theory: if you have the type of some function, then the set of possible implementations of that function is (usually) countably infinite, so the type doesn’t really tell you anything. But of course you probably care only about one of the simplest, so the type really does tell you an awful lot.
Anyway, I’m with you on the business of IQ tests. The problem is that as answers become increasingly open-ended, they are both more representative of a person’s actual intelligence, and more difficult to score objectively.
I think the simple fact the writer knew the answer, (Hen) but then went on to explain why it might be other things showed a lack of intelligence if anything.
Anyone who's intelligent would know Kosher is a cultural test so is not valid.
If you don't know how to answer the question correctly it's a lack of intelligence, not you're smarter than the test. Knowledge of the state of mind of others is what is normally defines intelligence in animals. Knowing what answer they want is part of the test. Some people just don't get it...
This is the same as psych tests that ask if you want to commit suicide. If you don't know why this question is there then the test is working. You're not smarter than the test because you think it's a stupid question, you just don't understand the test.
Note that a proper IQ test must be administered by a trained professional and takes a couple of hours. This is simply because there are so many pitfalls that there is no other way - anything else that calls itself an IQ test is a pale imitation and can't be anything but flawed, for all the usual criticisms. What the author describes may be what Mensa need for admittance, but it's not a proper IQ test.
For the hen question two answers must be accepted, and second one, based on the number of letters, must be rated higher.) At least if you want to catch a person with Asperger's.
Regarding Mensa itself, or any claims to intellectual superiority: one of the big lessons of my life has been that being intelligent has no intrinsic value. If you are unable to manifest your intelligence in a way that can substantially (disproportionate to your effort) improve your life or the lives of others, then your mind doesn't matter. It has no impact in the world, and I am not the first person to observe that having a powerful mind is at many times a burden.
You can see that other people may be more stupid than you, forwarding simpler arguments and relying on less rigorous thinking than yours, but if they don't want to be convinced of that, then you are powerless to make them see it. They go off blissfully, and you burn with anguish. Again, if you cannot manifest your intelligence in a way that forwards your agenda, who cares?
To phrase it in a more confrontational way: if nobody is forced to contend with your mind, nobody knows it exists. Or to summarize it in a quote:
"[Intelligence] is like being a lady: if you have to tell people you are, you aren't." -- Margaret Thatcher
This clicked for me during my final year of college, when a friend of mine was taking an LSAT preparation course. During a break, he was chatting with the instructor about realistic outcomes, and said earnestly that he hoped to get a score of 168 (which IIRC is a very high score.) The instructor, a man of maybe 30, who had been out of school for several years, scoffed and replied, "Good luck, I only got 165 and I'm a genius."
And what did the genius do when not teaching LSAT prep courses? Why, he worked at Target.
Edit: expanded a little on the first couple of paragraphs.