Hacker News new | past | comments | ask | show | jobs | submit login
How Much Does Education Improve Intelligence? A Meta-Analysis [pdf] (utexas.edu)
54 points by pseudolus on Dec 13, 2021 | hide | past | favorite | 130 comments



As a funded research academic who's studied cognitive ability(ies) for 25+ years, there are too many wacky and mistaken statements in these comments. Intelligence (aka g or IQ) is one of the best researched constructs in social science. We know more about it than virtually any other concept in the people space. We know what it predicts, we know alot about group differences on the variable, and -- to the extent possible in any science -- we know that the group differences are real and not a result of systemic bias in the assessments. We know there's a strong genetic component and, of course, enriching experiences like education can increase it to some extent. We know it's relatively stable from 21 to 60 and, at the group level of analysis, it decreases as folks move from middle age to old age.

So, this is one of those areas I know way too much about and many of the comments seem wrong-headed. Makes me wonder how much faith I should put in comments on topics where I don't know as much.


>Makes me wonder how much faith I should put in comments on topics where I don't know as much.

Despite murmurings to the contrary hn is no different from any other social media site that doesn't curate content. The fact that the denizens are techies or whatever doesn't supervene that fundamental fact. So the answer here is you should trust comments on hn just as much as you trust comments on Facebook.

As another funded academic in an area that comes up very frequently here (deep learning), personally I use hn as a link aggregator for interesting content and pay very little attention to the discussion.


I don't think we actually know anywhere near as much as that about it. We have no idea that there is no systemic bias - even progressive matrices tests can credibly have some bias, I found them a lot easier since I've learnt about XOR for example. This is not something that has been studied sufficiently and it's very difficult to know each way, it remains that IQ tests are a cultural artifact.

As far as a strong genetic component, there are very few studies that actually establish that and they all have considerable issues. The extent that we know is that IQ is relatively strongly heritable, not genetic.

You will find that many funded research academics that study cognitive abilities disagree with your assessments.


This is exactly the type of commenter he's talking about. Contributing to Gell-Mann Amnesia.


I'm repeating what people with PhDs in psychology and psychopedagogy with specialities in measuring knowledge, intelligence and ability, which are currently implicated in ongoing research in the field have told me, almost verbatim.

There are currently a lot of unknowns as far as IQ. We have yet to accurately measure the cultural impact of IQ tests, in fact, we have yet to come up with a solid methodology for that, as all IQ tests are cultural artifacts.

We have yet to properly measure the fetal and epigenetic components of IQ, with differing impacts found that are still not totally accounted from.

We have yet to manage to completely square away at the Flynn effect, which is an incredibly large effect on IQ that may be fully cultural according to some psychologists, or based on non-cultural environment, as far as I'm aware or have been told this is still not fully explained.

At the same time, changes in relative IQ scores between groups are difficult to account for if IQ is as heritable as it seems to be,

There are a LOT of unknowns in IQ. It is far from "as well as understood as science can". We don't even know all that we don't know.

This isn't the Gell-Mann effect in action. This is someone trying to pass off a controversial opinion in a field (perhaps not amongst people whose job is to work on the tests for obvious reasons) as the established truth, and in doing so ignoring a body of research, which is far from unusual especially in these fields. At the very least the fact that I've heard one thing from multiple subject matter experts and something else from them shows that there is significantly more doubt and uncertainty as claimed in the original message.


I hold an MS in psych, specializing in intelligence (the study of), and the original message matches with what I know. What you're saying matches with what people with anti-science agendas hold (and with what people who've received low scores on IQ tests say). Go back to Reddit is my advice to you.


That's a very poor ad hominem. I'll leave it at that.


The problem is that you’re not smarter than me so why should I trust your test?


> So, this is one of those areas I know way too much about and many of the comments seem wrong-headed. Makes me wonder how much faith I should put in comments on topics where I don't know as much.

Experiencing the Gell-Mann amnesia firsthand is always weird. In an expert in deep learning, a research topic that arguably is mich closer to HN's heart. Yet HN comments on the topic are usually a bit off the mark. They feel more like something people would say after taking a 101 undergrad class in the topic, rather than a truly informed discussion. A lot of my PhD work revolved around computational biology. And while I don't work in the area anymore, i know enough about it to see that in sich a topic that's further from HNs core, comments are often not just "a bit" uninformed,but utterly so


Do you study artificial intelligence?


I have seen this also in GWAS. I come here to talk linux and programmer stuff, politics is trash everywhere. Its a shame that any geneticist is seen as racist by the public. If you want to see a more exclusive site: https://lobste.rs/


It seems to me that possibly the highest return on investment a government can do is investing in education. More intelligent people likely impacts tax revenue, crime etc. does anyone know if there are studies done on this?


Education scales terribly because in every small town, for every subject, you need a teacher. The level of teaching varies widely, and teachers have no reason, other than ambition, for the quality of education they deliver to be top-notch.

The purpose of education is also very widely misunderstood. I often see (for context: I'm in Eastern Europe) regrets that the education process does not provide valuable life skills, that there is too much focus on subjects like physics or biology.

School teaches you to be systematic, to look for information, to know that even if you don't like or want to know something, you have to sit down and learn it. These are very important lessons that can later be translated into other "life skills".


Seems like education doesn't have to scale terribly - perhaps we should move to a system where the highest quality teachers produce content that can be asynchronously shared globally (think Masterclass, language permitting), while the "in-house" teachers can work more as "guides".

There'll still be a varied quality of in-house teaching, but at least the primary lessons could be world-class regardless of the location.


This has been happening for years. The highest quality teachers (for some definition of "quality") sell lesson plans to their peers.

https://www.teacherspayteachers.com/


It doesn't really work like that - you can produce and share exceptional resources, but that's not a substitute for a good teacher in the room.

Education definitely can scale, but good education scales much less well than slick-but-ineffective education.


"In-house" teachers are have been using asynchronous materials for decades now. Books were available since forever and videos for like two generations now.


> where the highest quality teachers produce content that can be asynchronously shared globally (think Masterclass, language permitting), while the "in-house" teachers can work more as "guides"

Yes, that would be best, but that removes the most fun aspects of teaching (curriculum development, lecturing) from the local teachers, while leaving them the most draining (keeping students on task, discipline, keeping the classroom quiet).


Like Khan Academy, which is already an invaluable tool for parents and teachers, but something is missing.


>> Education scales terribly

That's why doing it well requires more money. That's why parents need to be involved, which is why we need a better economic system where productivity increases actually lead to more free time instead of an increasingly Dilbert way of working.


> School teaches you to be systematic, to look for information, to know that even if you don't like or want to know something, you have to sit down and learn it. These are very important lessons that can later be translated into other "life skills".

School tries to teach this. The literature seems to indicate that it's not very good at actually teaching this.


If more people are trained you’d have more teachers available everywhere.

Starting at “artificial scarcity”/zero and building up/out is ridiculous.

There are plenty of people who would do such work, who probably need little training, but reliable routine blows up the speculation game. We exist in a state similar to watching a movie; suspended disbelief. These 60+ yo political truths? Don’t look away!

We know a whole lot more about logistics and manufacturing now than we did in the 50s.

It’s time for the story to move along.


Educated people require opportunities to use their brains though. If they don’t get these, they will not take manual labor, they will complain and then topple the regime.

Uneducated people with a nice conservative religion is the way to go.


It's really sad that we have to read thoughts like this in 2021, and makes me think what we need to do to help more people overcome selfishness.


Then start with not voting for people who run on these ideas.

(if you are lucky enough to have the privilege to select your government without a bloodshed)


Sialkot, Pakistan would like a word with you.


According to this Bloomberg report (https://www.bloomberg.com/view/articles/2015-01-23/spending-...) 10 percent increased spending leads to "children to complete 0.27 more years of school, to make wages that are 7.25 percent higher"


One of the most effective investments in education we can make is in preschool education:

> Although these elements can be expensive, “when well-implemented and supported by subsequent schooling, high-quality preschool can pay for itself. Studies of high-quality programs that have followed students into adulthood find up to $17 returned in social benefits for every dollar invested. This is because people who attend preschool are more productive in school, work, and society generally—with higher levels of education and earnings, less involvement in delinquency and crime, and fewer chronic health problems,” said W. Steven Barnett, Founder and Senior Co-Director of the National Institute for Early Education Research (NIEER) at Rutgers University.

https://learningpolicyinstitute.org/press-release/what-does-...


> It seems to me that possibly the highest return on investment a government can do is investing in education. More intelligent people likely impacts tax revenue, crime etc. does anyone know if there are studies done on this?

I disagree, you don't need to have a smart society, you can have a moral society and that takes away a lot of government spending for the unruly. An overproduction of highly educated people doesn't make a society better if there isn't demand for those jobs.


> investing in education

You really think the problem is lack of investment? Education is insanely overfunded, but it still fails to provide its basic function - that is, of educating its consumers.


If it’s insanely overfunded, how come a career in tech is many multiples of income vs teaching? Good luck buying a decent home on a teacher’s salary in attractive areas.

Inspire and grow the intelligence of the next generation? Nah, let’s sell ads and figure out how to make people spend more time on social media, getting rich in the process.


Strange, isn't it?

According to [1] the American people spend $14,891 per public school pupil per year. So if a single classroom has 30 students, that classroom gets $446,730 of funding.

And yet, as I understand things, American classrooms don't look like they've had $440k per year spent on them.

[1] https://nces.ed.gov/fastfacts/display.asp?id=66


Teachers salaries are mostly about right, and the 'problem' in America is generally not the education system to which most people who achieve to any degree have access ... it's social conditions.

This thing about 'amazing teachers vs. terrible teachers' is a really, really bad mythology.

Most teachers are decent and as long as they are paid reasonably well, the quality will not rise, they are not motivated by income. The very notion implies a lack of understanding of communitarian values.

In the worse neighbourhoods, kids don't show up, they are years behind, their kids are making money in gangs, they face violence and threats, they have no peers into academic stuff, they have no role models, they have no money, they see no reason to finish.

But the same school and teachers magically transplanted into an upper middle class neighbourhood will flourish.

America invests more as a % of GDP in education than most nations, even raw $ per student [1]

The other thing that nobody wants to contend with, is that America has a giant community of migrants, often undocumented, from very poor countries with a different culture that live in more segregated isolated communities, and form a kind of 'working class backbone', but which makes it harder for their kids to break into the middle class. The surpluses from this labour are immesurable.

California and Texas are rich at least in part because they have millions of people willing to work very cheaply in addition to millions who will work 'off the books' even more cheaply - and who do the work well because they are 'happy to be living the dream'.

PISA (OECD) standardized testing (see: 2018 and historical) shows this stark difference: Asian and White kids have done consistently well for decades. Black and Latino kids are consistently 'doing a bit better' (i.e. closing the gap), but the increase in their population offsets the net improvement and drags the average down.

The 'education' issue in the US is a giant canard. If you can get kids into good families, with good parents, stable jobs, no violence i.e. 'what you and I would normally expect in life' - those kids will do just fine in the current system as it is.

[1] https://nces.ed.gov/programs/coe/indicator/cmd


> Most teachers are decent and as long as they are paid reasonably well, the quality will not rise, they are not motivated by income. The very notion implies a lack of understanding of communitarian values.

> In the worse neighbourhoods, kids don't show up, they are years behind, their kids are making money in gangs, they face violence and threats, they have no peers into academic stuff, they have no role models, they have no money, they see no reason to finish.

These two paragraphs are mutually contradictory. A high-quality, community-minded teacher is not going to want to deal with "gang violence and threats" if they aren't being paid enough to make it worthwhile for them. They've got plenty of alternatives, after all.


You're thinking like a capitalist, not a 'communitarian' for lack of a better word.

It's hard for me to grasp how people can really even think that way for anything but careerism - I always felt that it would be the perspective of industry, not the the community at large.

Teachers love children, and want to help, that is their vocation and calling.

For most people, money obviously matters, but it's not everything.

Generally, speaking, they are going to view children in worse conditions as a challenge and an opportunity to help, not some 'crap to contend with'.

Surely a few teachers are going to want to 'teach the best students' but that's a smaller cohort.

More broadly, yes, it can probably be a bit depressing to work in crappy schools, but they do often pay a bit extra.

But ultimately, the point is moot, because even if schools do operate 'sub optimally' in the face of non-participation from students, those same schools would pick up as the students did i.e. if you solve for the community problem, the school will be up to the task.

Bad schools are not creating these environments, it's the other way around.

The model for education works fine, just get people stable jobs, stable families, conscientious lifestyles and educational outcomes will go up.


If you don't pay enough then of course the only remaining incentive is love of children.


It's not an 'incentive' it's a disposition.

Most people don't have 'careers' they have 'jobs' and if they can do something they want to do then that's a bonus for them.

So comp. takes a completely different kind of form there, outside the scope of what we normally consider performance/bonus oriented among professionals.

It's a bit like Nursing, a semi-socialized trade where we establish pay rates via budgets, Unions, a bit of supply and demand etc..


Because inefficient bureaucracies like education systems waste capital. They have no signaling mechanisms for success or failure. Notably, a teaching degree is the same price as a computer science degree, and some colleges are able to charge six figures in tuition even if they provide no better job prospects than community college or less.

As to why software devs make so much, it's a simple matter of revenue per employee. Software is insanely scalable. While it is capital intensive to produce software, it is nearly free to distribute. A handful of developers can produce multi-billion dollar software programs, because the market is the entire internet (billions of people). A grade school teacher can at most teach 30 or so kids at a time.


> If it’s insanely overfunded, how come a career in tech is many multiples of income vs teaching? Good luck buying a decent home on a teacher’s salary in attractive areas.

Engineers ship results.

Pretty sure many parents would be ok with a result based comp for their kids teacher.


When you find out that one of the biggest investors in tech is the CA teacher pension fund


When you consider if you were good at math in the 80’s being a math teacher wasn’t a bad job but now if you’re good at math you make double in tech….


Try more like triple, if this link is correct about starting teacher salary in SF: https://www.ziprecruiter.com/Salaries/Entry-Level-Teacher-Sa...

They're claiming an average of about $40k, and $63k at the 90th percentile, for an "entry level teacher." However, note, the shape of the distribution is really, really freaking weird, so... ¯\_(ツ)_/¯


> Education is insanely overfunded, but it still fails to provide its basic function.

I assume you're speaking of the US, apologies if that's not the case. While there's waste in the system I'd argue that we do not over spend on most students.

Let's start with the statistic that the average spend per student in the US is $15,000. This number is misleading, a more useful number is the median spend per student, which is closer to $11k[0]. It is not uncommon to see per student spend around $9k[1] (p32). The US is more unequal in its spending than most first world countries.

Perhaps more important, spending by many other nations is under counted relative to the US. Learning is highly dependent on overall health and feelings of safety. Food and housing programs, mental health services, etc. contribute substantially to educational outcomes. Finland, for example, invests heavily in early childhood mental health[2]. Budget-wise, this is mostly accounted for in their health spending whereas in the US, such services are generally accounted for under special education (~17% of school spending).

[0] https://aasa.org/SchoolAdministratorArticle.aspx?id=9248

[1] https://nces.ed.gov/pubs2020/2020303.pd

[2] https://www.tandfonline.com/doi/full/10.1080/00207411.2020.1...


Let’s distinguish schools and education. Schools include buildings, administrators, utility bills, IT, football teams, etc. Education is actual learning by students of important things they wouldn’t otherwise learn elsewhere (science, literature, citizenship). I am not sure schools are overfunded but funding schools is not the same thing as funding education. Education, as far as I can tell, is simply not properly incentivized.


So how do you determine that education is overfunded. In the US budget education spending was just over 10% of military spending in 2016 [1 ]. So if education is overfunded what is military spending?

[1 ] https://archive.attn.com/stories/11036/how-military-and-educ...


That's just the federal budget, and does not include state and local spending. Most education funding is at the state and local level, though.


The US spends significantly more on education than the military. K-12 education spending alone is the same as the entire military budget.

States pay for education, Federal pays for military. The Federal government isn't responsible for funding education, so it is unsurprising that they have a relatively small budget for it.


well, there's a world for that: colonialism.

It's a well known fact that US supremacy is mainly due to their military power, but, for example, they lack behind in many other areas, even compared to not completely developed countries like China (that invests a lot more in education[1][2] and cultural products[3])

[1] the United States spent 2.6 percent of GDP on total expenditures on education institutions

[2] public expenditure on education in China amounted to around 4.22 percent of national GDP. That value increased from around 2.5 percent in the mid-1990s to above 4 percent in 2012

[3] China is now the lead exporter of cultural goods, followed by the United States. In 2013, the total value of China's cultural exports was US$60.1 billion; more than double that of the United States at US$27.9 billion.


> Education is insanely overfunded

Can you cite a source for that claim? Genuinely curious, because I don't know if I've ever heard someone suggest it. (Conversely, the idea that education is substantially underfunded is generally assumed - at least in my bubble.)



So what do the US spend education money on? I would've imagined that teacher salaries would be the highest contributer to education cost, but for teacher salary the US is well behind everyone else.


In the US, I assume administration and materials (vendors for everything from desks to books). The US teacher's unions have been regressive for decades, wherein costs have gone up without the union being able to get better compensation. Ofc the employment benefits that teachers get is no sneeze. Retirement, healthcare+for your children until their 20s is something special (at least it was when my mom was a teacher).


US has 4th higher education spending per student in the world, just behind Austria and Norway (US and these two all spend around $15k/student), and behind Luxembourg ($20k).

But, national per student spending only really small part of the story. It is more instructive to observe that there are lots and lots of school districts which spend more than $15k, but nevertheless get atrocious educational outcomes, for example New York City, Baltimore, or Chicago. If New York City public schools can spend nearly $30k per student, but nevertheless have worse outcomes than national average, why would anyone believe that more money is what American schools need?


I commend Caplan's The case against education to all the downvoters. The above comment might be wrong, but how sure are you it's so blatantly wrong it ought to be obliviated?

https://en.wikipedia.org/wiki/The_Case_Against_Education


> Education is insanely overfunded

Those South Dakota teachers didn't think so.


The vast majority of the increases in education spending over the last decades have ended up expanding the state-level bureaucracies rather than making it to teachers and classrooms. Until we make some kind of change to the systems that ensured that outcome, we can expect the same to happen in the future.

The amount of spending isn't the problem -- we've consistently increased spending. But that increased spending doesn't increase classroom budgets or teacher salaries significantly.


You get more out of funding a good safety net so you don't send starving children to school and parents can spend time with their kids and get involved in their education.


Schools are that safety net, to some degree. They feed children and provide daycare, after all.


are you trying to agree or disagree with me?


That safety net is part of education. Schools keep children off the streets for 7 hours a day, offer reduced lunch. Investing in a social safety net is code for investing in public education.


I think it's dystopian that we give schools that job. Nobody should NEED to go to school to eat. And what about the parents?


The difference between theory and practice...

Education in the US has never been more accessible or more extensive, yet we have a massive student loan crisis ($1.71 trillion of debt) and we import more knowledge workers than we produce (~350k H-1B visas in 2016 and 2017).

Maybe we're just "doing it wrong", but we're not yet seeing the fabled "highest return on investment".


I think this depends on if people want to govern or simply win elections. If one's goal is the latter, maybe smartening up your base is not a good thing.


Unfortunately for the United States, the last Secretary of Education was the heir to the largest MLM scheme in the nation (possibly world?)

More intelligent people are likely a detriment to MLM revenue.


Let's be factual, she married into the MLM family riches. Her own family was super rich because of a patent which is a totally different scheme where you skim some wealth of lots of other people's work without really contributing anything to society.


That seems like an ungenerous characterization of patents. There are issues with them but I wouldn’t say that it’s primarily a mechanism to skim some wealth of lots of other people’s work.


That's literally the original historical purpose, and some would argue still effectively is, but yeah patents have a great PR team.


Fortunately for the united states school is funded at the state level.


Unfortunately for the US "funding" can mean "public money is channeled to corporate monopolies which own various markets for teaching resources."

It's also managed at the school level, which means all kinds of people with eccentric belief systems are motivated to take over school boards and force their issues on others.

But elements of curriculum design are also created semi-centrally. So you can get the worst of all possible worlds, with corporations skimming money from classrooms while faddy ineffective teaching techniques collide with school board skirmishes.


In Texas textbooks and curriculum is state level mostly. Idiots on school boards don’t have much to do with what is taught. And definitely not in important classes like math. And most funding goes to the schools not corporations


Education, yes... but schooling, no.


How does this square with intelligence studies of deaf people? Despite being half as likely to complete tertiary education, the deaf exhibit the same IQ distribution as the general population. Are they doing something else that is just as effective as education at increasing IQ?

> In sum, the implication of the research of the last fifty years which compared the IQ of the deaf with the hearing and of subgroups of deaf children indicates that when there are no complicating multiple handicaps, the deaf and hard-of-hearing function at approximately the same IQ level on performance intelligence tests as do the hearing. [1]

[1] https://academic.oup.com/jdsde/article/10/3/225/413353


Relatedly, playing a videogame will make you better at that videogame.

Whenever we speak of education and intelligence, you really have to remember the limits and inclinations of the tools you're using to measure the things. IQ tests are a very good measure of how good you are at IQ tests, and success in school is a very good measure of how good you are at taking other kinds of tests.

But concluding anything more than that requires something more rigorous than tends to happen in this area.


> something more rigorous than tends to happen in this area.

This is a great example of an ad-hominem attack, but against a whole profession, that tends to pop up fairly regularly on HN. I saw somebody else call it "hackers are better X" or something like that.

So yes, there is this whole profession, that do nothing else but debate the subtleties of said profession. That's what they do, they go to great length, setting up experiments, writing papers, and trying to outshine their opponents at conferences. They have been doing this for decades.

But they couldn't think if a simple counter-argument? How likely is that?


So, I work in this profession. I teach in academia for a living. And I don't think it's particularly controversial to say you can find lots of examples where yes, the whole state-of-the-profession is wrong for long periods of time.

There's nothing necessarily wrong with that, in a way it's kind of the point -- come up with ideas, beat them up a bit, see what's left, sometimes very little.


They have, and did, and many psychologists, psychopedagosists, and other people in the cognitive sciences agree with the OP. I don't see how they were being arrogant and snarky when the exact argument he said does indeed show up in academia.


There is no such thing as a test that is not an IQ test. That is why the term "g" is used -- it stands for "general". The thing people discovered was that performing well on an "IQ test" predicted that you will perform well on any general test. Whatever an IQ test measures generalizes to other domains.


"Exam technique" is definitely a thing, though. For example on a timed test, not wasting excessive time on a question you don't know the answer to.


It seems like scheduling and prioritization are important elements of intelligence.


Given a large part of society works and revolves around the dynamics of a school, if you do "good" in school you'll have a better change to do "good" in life.

It's very far from being an 1-to-1 relationship, but if you know and play the game of society and their institutions you'll be in the "system" with all the pros and cons.

The life of an outcast is both romantic and brutal.


sure to some extent. But I'm sure "good at iq tests" is also a good measure of other things. "Intelligence" is a bit of a loaded word, but it wouldn't be surprising for example if a group scoring >90% on iq tests, have higher scores on average on the SAT than the bottom 10% of iq test scores.

You can also see fairly strong correlation between profession and iq scores. Doctors and professional mathematicians score higher on iq tests on average.


When I studied psychology in university, the general thinking at the time was that IQ tests measured education and not actual intelligence.


This is very strange, especially as stark differences in measured IQ (and, in fact, in intelligence) between small children are often rather apparent, before one gets any substantial amount of formal education, and because there are no known studies showing that education can actually increase IQ in any non trivial and lasting manner, other than literally teaching people answer to specific IQ tests.

Why would psychologists believe something so clearly wrong?


All IQ tests do measure things that people literally do teach themselves. From finding synonyms and antonyms, to finding the center of mass of a shape, to XORing two bitmaps, etc..

Research has found that differences in IQ at a young age are not very stable over a lifetime. That being said, education starts at birth, and cognitive development doesn't go at the same speed for everyone even when the endpoint is the same.

And actual, we did find that education does raise IQ in a significant and sustained way, with measured increases persisting for 40 years : https://journals.sagepub.com/doi/10.1177/0956797618774253 (ie, the OP)


>Why would psychologists believe something so clearly wrong?

I would imagine from their research? Why are you so self-assured that you are right?

Often the issue with 'IQ' tests is that the way they are written introduces biases. Any definition of intelligence is at some level socially constructed, which brings social biases into play. Designing accurate, fair, and reliable tests of any educational measure is difficult. Doing a generalized one is even more so.

really obvious example: I write a question that references a baseball game, some people have the tacit knowledge that there are nine fielders and one batter...some don't. Am I measuring education? information gained? 'intelligence'? Is someone from India who makes an analogy to cricket more intelligent than someone who doesn't? IQ tests have generally been shown to mostly be predictive of...scores on IQ tests.

Fundamentally, when dealing with people you are dealing with latent constructs. They, by definition, cannot be measured directly and we presume they cannot be measured exactly. The issue in this paper is they aren't even really defined - they measured cognition not 'intelligence'.


You speak as if these issues have not been studied to death a generation ago already. Jensen’s “Bias in mental testing”, a nearly 800 pages volume full of references, covers it rather extensively, and it came out 40 years ago.

What makes me self assured here is that best counter arguments to the body of scientific research I read on this topic that I hear is people musing about some hypothetical IQ tests that might possibly ask about baseball. I mean, consider Raven’s progressive matrices. It’s not the best IQ test (for reasons other than knowledge bias), but it correlated very well with WAIS and Stanford-Binet, two most popular and arguably the best IQ test. It asks for no knowledge of baseball, regattas, or anything else, in fact, it used no words at all. How comes that it nevertheless gives the same results as other, allegedly biased IQ tests? What exactly makes it biased, and why in exactly the same direction as other tests?


It doesn't give the same results as other IQ tests, it's correlation is nowhere near as strong as you let on, the average expected difference is 2/3rds of a standard deviation.

Beyond that, yes, RPM is culturally biased. If you have experience with basic geometrical manipulations and bitmap manipulation you will find it much easier. For example if you have CS training you will notice that half or more of the questions in many RPM style tests are basically XORing two images together, with a small twist some times.

You'll also find it difficult to argue that that people who culturally do less manipulation and work on paper or on a screen will have a more difficult time.

Beyond that, there is clearly a body of research that finds IQ tests to be culturally loaded, and solid research that shows that even further studies in the same society where the IQ test was developped has a significant and measurable impact.


LSATs and verbal reasoning tests are very obviously culturally informed. Even so - questions about baseball games are very much not something you'll see in an IQ test.

But it's not a simple issue. Tests like Raven's Progressive Matrices looks like they should be culture-free but in fact they're based on assumptions, contexts, and operations that are taught implicitly in basic math classes in the West.

If you don't have that education, you're going to find it hard to understand the test.

Other culture have different ideas about what intelligence looks like and how it's expressed. If you threw equivalent non-Western challenges at a supposedly smart Western kid it's not obvious how well they'd do.

https://www.apa.org/monitor/feb03/intelligence


totally agree with this post, I used a simple example where many more complicated ones exist.

The problem is how do you disentangle Differential Item/Test Function being from cultural purposes as compared to 'we expect this group to perform worse'.

The larger point is that culture, intelligence, and measures of culture developed in a particular culture are difficult to disambiguate.


Indeed. After taking a progressive matrices I found that half of the second half was just XOR on bitmaps.


Tell me the biases involved on tests like this: https://www.mensa.org/public/mensa-iq-challenge

IQ tests are incredibly predictive of a whole host of outcomes.


well for starters: the test instructions are in English - which doesn't seem relevant to the test, it focuses on abstraction as a modality of measuring cognitive capability, test taking, and computer user interface knowledge.

I work with high performing undergraduate students (think top 10 US public university). In one of our classes, they work on global public health design projects. Last year, the project was centered in the country of Chad.

One thing the students learned was that if someone isn't part of a culture that regularly uses paper/2D representations of things then using those modalities to communicate prototypes doesn't work.

Does that mean my students are smarter than the Chadian people? Why? Because they come from a culture where interacting with abstractions and representations of information is normed? The people of Chad certainly can.

If they are smarter shouldn't I be able to drop my students near Lake Chad and let them learn how to navigate without a map and compass? its just cognition so they should be able to figure it out right? They could memorize the map in advance?

It's not that 'some things have bias' its that all things are socially constructed at some level...


Right, but that was just an example of the types of questions on most IQ tests. The one I was given at a psychiatrist's office had no words or even letters for the multiple choice answers.

As far as dropping your students near Lake Chad, that's not general intelligence. Babies born in Chad learn how to navigate their area, just as your students can. The question would be who would be able to learn how to do it faster, and how many of each group could learn more complex things.


So what is general intelligence? And why does it always seem like definitions of general intelligence just happen to align with the current hegemonic culture?


Hint: it has to do with uncomfortable differences in IQ testing between certain groups of people.


The differences have been proven to be due to socioeconomic factors, not biological ones.

But I'm guessing that fact makes _you_ uncomfortable.


Nope, I'm certainly open to that and in fact I would love for that to be true. It's far from "proven" though. Most evidence points in the opposite direction. Adoption makes for some great data. Of course, good luck doing any type of legitimate research on that nowadays without being cancelled.

I understand why it's something that we've collectively decided to ignore, but my entire life my state has also been throwing money at the "achievement gap" to no avail. You can't help fix a problem when you're coming at it with wrong assumptions.


Are you talking about black kids? Because I can assure you that no one gives a single fuck about their feelings or any uncomfortable differences.


> Why would psychologists believe something so clearly wrong?

Because they are scared of the slippery slope in questions of nature vs. nurture. Steven Pinker's book 'The Blank Slate: The Modern Denial of Human Nature' explores this topic and the cultural antecedents of naive nurturism.


I have to wonder: how much of a factor is social pressure in this setting? Any hypothesis geared around IQ being anything other than something that is malleable with education seems to be "too dangerous to handle" for any serious scientist.

I am not a social scientist, but based on my experiences seeing people attempt to learn programming, it seems like certain types of thinking patterns cannot easily be learned by everybody.


It's worth noting there are good reasons to believe the effects here are somewhat spurious – not "true" improvements but hollow gains akin to "teaching to the test." For instance, the top display of figure one suggests a fade-out effect with age, similar to the one seen with intelligence gains from pre-school programs (which disappear within a few years) [0]. Also, there's a reverse causality issue the authors don't fully control for, in the sense that more intelligent children are also more likely to persist longer in schooling.

As more and more careful studies were done on early childhood interventions, with larger sample sizes, the promising results of early investigations vanished. I expect the same will (largely) happen here.

[0] http://www.johnprotzko.com/wp-content/uploads/2015/11/Protzk...


When I learned about early education a long time ago, the argument was not about boosting short term cognitive abilities, as in this study about IQ tests, but long term socioeconomic improvement (i.e. we heard about changes in graduation, employment and crime).

https://equitablegrowth.org/wp-content/uploads/2021/09/playf...


Yes, I think this is a much better frame on the research. (And one that e.g. James Heckman has switched to.)


This is a meta-analysis. It inherently has a selection bias of other studies which failed to show a null-effect.


These tests are probably measuring something. But I don’t see how anyone with a high IQ can take IQ test seriously.

I’ve been studying the writings of early artificial intelligence pioneers (Minsky, Simon, Newell, Von Neumann, etc) and to think that we can assign an integer measure to the most complex information processing systems in existence seems like one step above pseudoscience.


Granted that measure is normalized, and my anecdotal evidence does match the feeling of being several standard deviations from the average person.

Maybe my gripe with IQ is that as a theory, I find it intellectually unsatisfactory. I can accept that being good at solving the puzzles they gave me in tag class correlates with positive life experiences.


Some interesting quotes from the discussion section:

> this design should be interpreted as producing a “local average treatment effect” that might not generalize across the full educational range (Morgan & Winship, 2015, p. 305).

> the key assumption — that the age trend extrapolates — is difficult to test. Moreover, although this approach produced large effect-size estimates, we did not identify any studies that tested whether these effects persisted into adulthood.

> The finding of educational effects on intelligence raises a number of important questions that we could not fully address with our data. First, are the effects on intelligence additive across multiple years of education?

> Second, are there individual differences in the magnitude of the educational effect? One possibility is the Matthew effect (Stanovich, 1986), whereby children at greater initial cognitive (or socioeconomic) advantage benefit more from additional education than those at lower advantage.

> Third, why were the effects obtained from the control-prior-intelligence and policy-change designs ... still apparent in later life, when effects from targeted educational interventions, such as pre-school, have tended to show fade-out into early adulthood?

> Fourth, which cognitive abilities were impacted? It is important to consider whether specific skills ... or general abilities—such as the general g factor of intelligence—have been improved

> Fifth, how important are these effects? ... to our knowledge, no studies have explicitly tested whether the additional IQ points gained as a result of education themselves go on to improve these outcomes.

> Finally, what are the underlying psychological mechanisms of the educational effect on intelligence? Ceci (1991) outlined a number of promising pathways, including the teaching of material directly relevant to the tests, the training of thinking styles such as abstract reasoning, and the instilling of concentration and self-control.

The discussion section is well structured to emphasize the confidence is purely in the correlation between education and an improved score, but there are still a number of open questions about the psychological mechanism of education, the socioeconomic and long terms effects, if effects are additive, and whether or not a change in IQ is meaningful and improves real outcomes.


IQ is normalised by age (apparently, it seems hard to find simple info on this) which suggests if you score 100 you are average, for your age.

In a society where nearly everyone goes to school, that seems like a confounding factor. Every year older will mean, for most people up to age 16 or so, another year of education.

Feels like you could pull some interesting info out of that, but not sure how.


TL;DR:

“we found highly consistent evidence that longer educational duration is associated with increased intelligence test scores. […] the results support the hypothesis that education has a causal effect on intelligence test scores. The effect of 1 additional year of education […] was estimated at approximately 1 to 5 standardized IQ points.”


Isn't there an ongoing debate whether or not IQ actually correlates with intelligence though? A skeptic could say that education focuses on mastering test-taking, so would naturally lead to increased test scores.


IQ scores tend to correlate with IQ scores...whether you think they are representative of a holistic intelligence is somewhat up to you.

These researchers looked at studies of cognition specifically, and did very little to control for the variety of measures of cognition that are out there or their disciplinary or general context from what I can tell.


From what I’ve seen the only people that think that don’t have any idea what (most) IQ tests are like. People seem to think they’re like the ACT, with questions that you read and can be taught. Instead you’re just completing patterns of symbols.

It’s yet another scientific facet of our lives that’s been politicized.


What? No. There are many kinds of IQ tests. Some require you to complete patterns of symbols. Most do have a verbal logic and purely verbal dimension, which is sometimes worth just as much as completing patterns.


You're right, I was too absolute in how I worded that. That said, I believe most widely used tests at least have a nonverbal portion. It's kind of a shame that we lump all IQ tests together; it's probably pretty easy for researchers to confirm their biases by using an inappropriate test.


Really? I thought they were all culture neutral raven matrices.


"Intelligence" is a somewhat nebulous term and people can disagree on what counts as intelligence. IQ measures something that is important for both education and life outcomes, though.


I was taught in my psychology 101 class that IQ is supposed to test innate intelligence: something that does not change over time or by learning.

In that context, this study literally is non-sensical.


The IQ tests are not designed by gods, and while this is not the intention, education could very well have an effect on IQ.

In fact that's the conclusion of the study here.


There's no such thing as 'innate' intelligence in the sense of not changing over time. Trivially, your IQ is benchmarked against a norm of people your own age, with the absolute value of correct results changing considerably with age (typically rising steadily until ~25 before declining). There are also techniques like the dual n-back that show at least short term changes to IQ tests.


But brains are plastic and ever changing, learning new skills strengths connections and forms real physical changes in the structure of the brain. For example someone could start out with terrible visual spatial skills, but through practice their brain adapts to improve so that it does not need to constantly keep expending do much energy on a task it regularly encounters.

There is no reason to believe that regions of the brain often tested in IQ tests can't develop overtime increasing ones score. Sure it almost never happens since most people never engage in activities that would strengthen it, but it's silly to think it can't.

Innate intelligence assumes the brain is static and cannot change, which has been proven to be false.


> innate intelligence: something that does not change over time or by learning

Isn't it a contradictory definition? If someone is intelligent then it will learn to improve on a test, otherwise, it's not intelligent. Maybe we should take the gradient of the IQ scores.


Nonsensical or a direct refutation of the idea?


Yes I suspect Goodhart’s law is unavoidable to some degree.

That said, are there really any other viable ways to measure intelligence in a standardized and reproducible manner?


The bigger studies in the meta analysis seemed to show a smaller benefit to education. Even so a few years of education adds only a few points. The gaps between like physicists and people in remedial math would require decades of school to close


Dumb question but can someone explain why we think that (1) education improves intelligence, rather than the alternative conclusion that (2) the intelligent are more inclined to stay in education institutions longer?


While the question is not dumb, the answer is in the very first sentence of the link. So aking the question here is, yes, dumb.

I know nobody read the links anymore and react only to the title but, seriously, that’s worrying to not even try when you have a question. Writing the question to ask probably took 10 times more than clicking on the link and reading the first sentence.


"Read the paper" isn't exactly the answer I'm looking for, but thanks for trying.

Only upon reading the paper do you discover that a "school-age cutoff", much like the famous hockey player study cited by Gladwell in Outliers, is one of the ways for controlling the selection effect I mention.


Read the article, that’s one of the first possible issues they address.


There's no reason it can't be both at the same time: IQ helps for education and education helps raise IQ.

Since they are controlling for the other effects, that means this analysis doesn't measure them. (Assuming, that is, the controls worked.)


They did point that out in the abstract. The data sources being school policy changes and school-age cutoffs does probably help a lot to prove the direction though.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: