It's common for people who received good educations to say that test scores aren't an acceptable indication of learning. It's probably true that if you already have an acceptable level of education, test scores shouldn't be too important, but it seems like the people criticizing test scores haven't experienced what education is like for the children being "left behind."
In middle school, the last three weeks of each semester were spent watching movies (the same two or three movies over and over). In high school, I was taught that the big bang theory was when the continents divided. I was taught that the most direct route between two points on the globe is through the north pole. My Spanish and woodworking teachers taught math because they knew that the math teachers weren't doing it.
When you're talking about schools like the ones I went to (which weren't nearly as bad as many other schools in America) I think that test scores are a decent indicator of the education a student is receiving. You're measuring things like "can this kid read English" and "can they multiply two numbers together." Standardized tests are certainly capable of measuring things like that. Testing probably breaks down for measuring intelligence/education as the level of education improves, but I encourage everyone to consider just how dysfunctional many schools are before ignoring testing entirely.
Except that even those things don't predict later success. Any kid with a family that can afford books will score higher on "can this kid read English". Any kid who's been drilled can learn to multiply two numbers together. Without cognitive skills behind them, though, those are not useful abilities.
The best predictors of future success are self-control, emotional intelligence and a grasp of functional algebra (which I believe, but don't have evidence to support, is because of the importance of computational and systems-level thinking.) Only one of those, self-control, is even indirectly measured by current standardized tests.
> Without cognitive skills behind them, though, [multiplication, reading] are not useful abilities.
Maybe I misunderstand you, but I fail to see how multiplication or reading are not useful skills. Good reading comprehension will help you navigate life in general, which is something lots of people struggle with. Similar for multiplication.
Even if someone is not very intelligent, if he's mastered a basic skill like that, he'll find lots of places to apply it. That's why there's so much emphasis in school on them.
From my work teaching I am very skeptical of the notion that it is a family's ability to "afford" books that makes the difference. It is whether the family knows that books are important and chooses to get them and read with their children. While monetary poverty is horrible, it's effect on educational outcome is dwarfed in comparison to cultural dysfunction and despair.
Yeah, the monetary poverty theory doesn't hold up when it comes to the case of poor Asian (or African) immigrants. Although it certainly plays some role when all other things are equal, I think the major issue to be tackled is the glorification of thug life and demonization of education in contemporary African American culture.
Sorry, that was unclear: the study I was thinking of specifically looked at giving poor families children's books and founded a measurable improvement in English skills. Of course access to a library that is open when a parent doesn't have to work, the time and energy to read to children and many other factors also play a role in early exposure to literacy.
> Any kid with a family that can afford books will score higher on "can this kid read English".
True, but there are also many families that can afford books but don't buy them, instead spending money on other things. Not having books in the house has been shown to correlate with decreased academic success.[1] I'm not saying this isn't a wealth issue, but that probably isn't the only dimension.
I think it's worth emphasizing that the research done in this article doesn't look at kids' absolute test scores, but only the difference between test scores before and after the school year.
Also, my school emphasized self-control, but how do you learn emotional intelligence?
While it is a relatively recent topic of interest, surprisingly it doesn't seem particularly difficult to teach. Much of the research is still in the preliminary stages, but one study found persistent improvement after a brief four-session intervention: http://orbi.ulg.ac.be/bitstream/2268/30253/1/Nelis%20PAID%20...
It will take further work to associate learning such skills with the positive outcomes associated with the skills themselves and figure out what the best ways to teach the skills are, but current research is promising.
Plus, books are really really really cheap. Go to any Goodwill or Salvation Army or Once Upon A Child and you will find gobs and gobs of books at very cheap prices. Yes, even undamaged.
We toss lots of books in the trash in America, not because we hate books, but because we make so many of them and then need to make room for the new books.
"Able to afford books" was once a status symbol. It's not any more.
This was more a correlation than causation (see Freakonomics for a description). Typically, families that can't afford books have other general issues that preclude their children from doing well (for example, they are poor, their parents typically work more than 1 job and can't spend much time with their children, etc).
I went to the University City School District in St. Louis. It was actually a pretty strange place because it's the main district that Wash U (Washington University in St. Louis) is in, so about 10% of the students are children of (mostly white) professors and other very education-oriented professionals (doctors, lawyers, etc). Because of that, top classes (honors and AP) were actually pretty good. I, and many of my classmates went to Wash U and other top universities.
The rest of the student body was made up of poor, mostly black students who came from families where education wasn't a priority. Every single year I was in high school the state tried to revoke our accreditation and we just barely passed a state audit to keep it. I was a math tutor, and it was common for students to come in who couldn't do basic addition (in high school).
To make matters more confusing, the gap between the good students and the bad was almost entirely caused by economics (wealthier parents resulted in smarter kids, as is normally the case) but because the economic divide was almost identical to the racial divide, many of the educational issues were easily conflated with racial issues. This led to the (mostly black) teachers and administration demonstrating favoritism toward white students, and it also created tremendous social pressure on black students not to try to succeed academically (otherwise you're an oreo).
I wouldn't trade my experience there for anything. I received a pretty good education, and I experienced a part of America that more privileged kids should experience, but most of the students at my school were definitely being left behind.
I can co-sign for the quality of the district he mentions. My nieces and nephew are there now. It's totally an economic issue. My sister-in-law really does care about the education of her kids and their potential future, but she has to work 2 jobs to barely support them adequately. It's a case of survival knocking education down a notch on the priority scale.
What class were you in to be taught that incorrect stuff from your OP? I can't imagine those professors and doctors would let it slide if their children were being taught such nonsense.
The honors and AP classes were generally pretty good (in math and science anyway) but those weren't always options. For example, in 10th grade, there wasn't an honors world history class, so I had to take the normal class. That's where I was taught what the big bang theory was. I knew it was wrong and tried to argue, but the teacher was pretty sure she was right. I immediately went to my adviser and got switched to a different teacher, but the rest of the class didn't know any better (many of them probably thought the teacher was right).
As for parents not getting involved, I'm not sure I ever told my parents about that stuff because I thought it was normal. Middle school was by far the worst part of the district, and a few years ago my brother and I were telling my parents stories about how about once per week there were school-wide riots in the lunch room that the FAs (bouncers) couldn't contain and we could do basically whatever we wanted for 30 minutes because of the chaos. It turns out my parents had no idea this was happening (my mom said she wouldn't have let me go back to that school if she'd known). We never told any parents because we thought every school had riots like that.
I have no idea how it happens, but I recall having more teachers that I referred to as "Coach" instead of "Mr.", "Ms.", or "Dr." while in middle school and early high school.
I'll add to this since I think these types of stories occurs more often than people think. I went to Baker High School in Mobile, Alabama (that might be your first clue).
In the 8th grade I took a required course called Earth Science, also of course taught by a coach. I recall that the last question of one of the exams was "The Universe was created by _____________". Odd. I thought this was a fairly complex answer for a simple fill in the blank question, especially for tests which were typically always multiple choice or fill in the blank. So I just assumed that the empty space below it (since it was the last question) was for writing a short answer.
So, I did my best to mention the Big Bang Theory and associated information about red shifts and such for scientific evidence. In the end, my answer was about a paragraph long, and I felt really good about my answer.
I did get full credit, but I noticed that other students also got full credit but wrote only in the blank itself. So I checked with the teacher for how I could have been more concise. Turns out that the correct answer that he was looking for was "God".
I guess the question was a simple fill in the blank after all.
I've wondered that myself, particularly in regards to the recent evolution vs. "intelligent design" debates in school as of late. I'd be interested to here what other HN readers think.
This is the most dramatic instance I experienced, but in Honors English we spent at least a week on the Bible, but never any other religious texts. Conveniently, at the same time I had read some vampire fiction by Anne Rice and concluded that it's possible that Satan is actually the hero of the Adam & Eve story. It turns out, however, that this is not a good opinion to express when trying to do well in English in a conservative public state school.
As to Earth Science, I was not penalized for my response; had it been, this would have been clearly objectionable. On the other hand, at the time I did feel that it was unfair in that one could give what I considered a cop-out answer for full credit. It may be a theologically true response, but it does not give us any scientific insight and understanding into our Universe.
Given the two examples above, I'm not really sure. It can certainly be uncomfortable for someone who is brought up as an atheist, but I'm not sure it's outright illegal. And this is where the avenue for standardized testing comes in -- it's not necessarily to demonstrate student learning but perhaps rather to better identify egregious issues in school districts.
> It can certainly be uncomfortable for someone who is brought up as an atheist, but I'm not sure it's outright illegal.
I'm quite sure it is illegal according to federal law. The question is whether state law trumps this. Because if it does, no doubt Alabama has passed a law making it legal to teach creationism in schools.
Even at good schools with non-religious teachers, students are undoubtedly taught lots of information that's false. Teachers are of merely average intelligence and often just don't know that what they're teaching is false.
Which is fine because if the subject matter actually matters, there will be ample chances to relearn it later. Having teachers teach stuff that is wrong is only a big problem if you think teachers need to be infallible. But the fact that teachers can be wrong seems to me one of the more important true things kids learn in school.
I'll have to look at the prior reports, but this 3rd report is somewhat disappointing in terms of its statistical rigor.
They do provide weightings based on optimal predictions for maximum accuracy with predictions for state tests scores, although those weights appear to assume a simple linear correlation. Even then, the data suggests that in class observations are almost a complete waste of time (single digit weightings), and that student surveys decline in value as the kids mature (no shocker: with younger kids, how much they like the teacher has a much more profound impact on their success at learning). In all cases the kids seem to be a more reliable indicator than the observations of professionals! This may just be an indication that assessments based on more frequent observations are more accurate, but the only way we can really get a sense of that is if they also had peer observations from team teaching environments... which are completely absent.
That's not the worst of it though. The worst is the rest of the data. It is based on some fairly arbitrary presupposed models for combining these different assessments. No breakdown of the individual components of these assessments (other than "English" & "Math") and no attempt to discern a model from first principles.
There are some other factors as well. Aside from some outliers, the accuracy of predictions seems kind of suspect. Most of the outcomes are within +/-0.05 standard deviations, and a majority of them are within what looks like +/-0.025 standard deviations. These are pretty minuscule variances (0.25 was supposed to represent a year of schooling, so 0.025 represents maybe 15 school days worth of progress at best), and it is not hard to see that without some of those outliers, the regression line doesn't look so pretty (particularly for English). I also didn't see any metrics on the variance of the data within a classroom, which I suspect makes any predictive value look limited at best.
Ugh. I really hoped that Gates would make sure there was more statistical rigor with an analysis like this. :-( I hate to say it, but the conclusions of the study are not well supported by at least what is in the final research report.
Yes, this is also the same foundation that spent 1 billion to make schools smaller, because they didn't do simple statistical analysis to find that smaller schools showed much larger variability. Now they're coming back to say "oops, we were wrong."
I can't say whether the findings are correct or not, but I like this:
>> The teachers who seemed to be effective were, in fact, able to repeat those successes with different students in different years, the researchers found. Their students not only scored well on standardized exams but also were able to handle more complicated tests of their conceptual math knowledge and reading and writing abilities.
To everyone who keeps saying "standardized tests are bad", please cut it out. A particular code benchmark may be useless, but benchmarking overall is good. Existing standardized tests may be bad, but in general, having some way to make objective measurements and adjust accordingly is the basis of all human improvement.
And yes, we should reward good performance, if we can determine what good performance is. How else do you expect to get more of it? It is incredibly difficult to make incentives that actually do what you want rather than encourage gaming of the system, but that doesn't mean we shouldn't try.
In short, let's stop debating whether we should measure and incentivize, and start debating how.
I grew up in the UK in a school where the emphasis of our entire education was "the test." Teachers entirely forgot how to teach a subject and our whole time was taken up with what essentially amounted to "test prep."
I did pass their tests, but did I get a good education? Frankly, no, I don't think I did. I mean statistically I got a great education, but it seems we have forgotten why we even educate to begin with...
Do we educate so people look smart on some random league table? Or do we educate so people are actually more well rounded, have a better grasp of the world around them, and do better as adults?
Or do we educate so people are actually more well rounded, have a better grasp of the world around them, and do better as adults?
I think we're all for this type of metric. The problem is that no one has yet been able to show a teacher's ability to achieve that.
So you're against testing. What metrics are you for? Do we at least agree that without any sort of metric to judge a good teacher vs a bad teacher that the whole educational system suffers tremendously?
I don't have the answers. I'm not even against testing. I've just seen the kind of damage overemphasis on testing can do to education and want to see it end.
When tests become a core part of how you judge someone's worth as an educator that educator is going to put all of their efforts into better test scores.
The question is do test scores mirror the quality of education or do test scores just tell you someone's potential at performing well on a test?
Testing is just one part of a three-pronged approach that the foundation is recommending. The other prongs involve classroom evaluations and student feedback.
I'm not sure that the argument is even that existing standardized tests are bad so much as that they are given too much priority. They should not be the only or primary means by which teachers, students, districts, etc are benchmarked (and in turn, rewarded or disciplined). "Teaching the test" is the problem, not necessarily the tests themselves.
Then why not use standardized tests for job interviews, or why don't decent universities use them? Standardized tests have serious disadvantages to consider, and aren't normally considered best-by-default.
There at least a couple obvious groups qualified to rate educational quality: teachers and students. If they happen to choose a standardized test to evaluate the education, then fine. Otherwise, I don't see standardized tests as obviously virtuous.
Don't confuse the government-mandated standardized tests with the SATs. They're not really out there to measure individual students. These tests are to ensure individual schools, districts, and states are giving the proper instruction to their students. They exist to make sure that Springfield High and Franklin High are both teaching the students the required material.
Universities do use standardized tests: they use the SATs and the ACTs, the GREs, LSATs, MCATs, Miller Analogies. They use AP exams to determine who gets college credit for courses the school never administered. They use their own standardized entrance exams for placement.
Further, many foreign nations (e.g. Japan, I think Singapore) have admission systems comprised entirely of standardized tests. Top 300 scorers get into college #1, 301-600 get into college #2, etc.
Then why not use standardized tests for job interviews
Because the government said it was illegal. (I'm simplifying, but look up Griggs v Duke.)
However, an employer can look at your college, and your college definitely can use your standardized test scores. So employers do use standardized tests, it just has to pass through a socially accepted filter first.
The problem with benchmarking is that it assumes all students are more or less equal. A measure that finds repeatable success for, say, 2/3s of the students will be model that will be emulated through the entire system to the detriment of the remaining 1/3 of students who do not learn under that environment.
Students are not a commodity. What works well for one person does not necessarily work well for another. Without focusing on the individual, you are always going to leave a group needlessly behind.
There are many important factors which teachers (no matter how good they are) can't control:
1. How wealthy, poor, educated, or uneducated a child's parents are.
2. How many resources the school and teacher are given.
3. Whether a child's parents are abusing them at home.
4. Whether a child is witnessing violence, hopelessness, despair in their neighborhood.
5. The attitude of a child's family and friends towards education.
6. Whether a child's parents are active in their child's education and in a PTA.
Study after study has shown that these factors greatly influence a child's attention and attitude to school work, their attendance, and ultimately their performance at school.
No child's performance on a standardized test, and no evaluation of a teacher's performance is going to pick up on the above factors, yet they can critically impact how well the child ultimately learns and what the child learns.
Standardized testing of a child that is unfortunate enough to live in a situation where the above factors are against them will likely notice a poorly performing student, one who gradually does worse and worse on the tests until they finally flunk out.
Then, instead of getting extra aid, the funding to the school that has a lot of such children will be cut (thanks to programs like No Child Left Behind), and their teachers penalized.
These factors, which are strongly associated with poverty, are things a given teacher is in no position to influence, but which society as a whole could address. If it has the will.
And at my job, there are many factors out of my control. I don't control the home life of my engineers. I don't control my customers' decision making processes. I don't control my competitors' actions in the market.
But yet I'm judged on the small slice of that overall pie that I can control.
What this study indicates is that there are teachers who can reproducibly create success levels given the control of the situation that they do have.
How is it any different than the rest of the working world where you can control some things and you can't control others yet your job is to do the best you can with what you can control?
Here is some experience that I have had. In an inner city high school (math class), in each mid-thirty student sized class, there were about four students who made it their point to disrupt the classroom. Unless actual violence occurred, they were not to be kicked out. So the teacher could spend a large portion of the period trying to reign in a couple of students at the expense of the whole class. Add to that that the majority of the class had difficulty with, not just multiplication or division, but SUBTRACTION.
This was an Algebra class, and the students had to be graded on their Algebra skills and judged on their Algebra proficiency tests. No homework would be completed, and no recourse could be given for lack of homework. 50% of these 9th grade students will not make it graduation. 3% will make it to a four year institution. Less will graduate from there.
How is this different than working with professional engineers? Ones who could be fired or in some other way worked with? How is this different than working code where a solution exists? How is this different than working with other people who are looking for a solution as well?
A better comparison would be to how you judge a prison guard in relation to recidivism rate.
That's a very legitimate question. How can teachers do the best with the resources they're given and the the children they're working with, no matter how threadbare the resources are, and no matter how traumatized and disadvantaged their students are?
Whether standardized tests have a role in answering that question or in evaluating teacher performance is hotly debated right now.
But the purpose of my original post was to draw attention to a different question: How can society as a whole ensure that every child gets the best education possible?
In schools that have no problem attracting resources, great teachers, and children with mostly fortunate backgrounds, perhaps this second question can be answered by asking the first: how can we improve our teachers?
But in schools that lack resources, whose students live in abuse, poverty, and despair, what even an exceptional teacher can do will be radically limited, and society would probably be better off focusing on how to eradicate poverty -- the real cause behind the failure of too much education in this country.
Unfortunately, seriously tackling the issue of poverty is much harder (and even contrary to some dominant American ideologies) than simply demanding more standardized tests be given and blaming teachers and their unions.
I grew up in two very different places; Bethesda, MD and Winthrop, ME. The former is a fairly wealthy suburb of DC. The latter is an extremely rural mill-town in Maine.
I know that the funding the schools get is blamed for a lot of their problems, and I understand where people who talk about how hard it is to teach kids with behavioral issues and histories of abuse are coming from.
I sympathize, but I also think that's a cop-out. I had bad teachers in both places. In Winthrop, I had horrible teachers. They took away advanced classes because an outside contractor thought it created elitist attitudes and putting smart kids in the regular classes would boost overall performance.
I had science teachers that just had us play with legos all day and gave A's to all the basketball players. I had math teachers that would lose assignments and blame the students. These were not well-meaning teachers that just had it hard trying to teach these poor kids. These were people that shouldn't have been teaching at all, but could not be replaced.
My parents actually went up to the school a few times to try and talk to the administration and teachers about what was going on. A lot of people did. They got push-back and were asked not to meddle.
There were 2 great teachers there and that's why I didn't completely screw up my life at that point. I've spoken to a lot of people with similar experiences; 1 great teacher that pushed them to be successful. That's why it's so maddening to me when people say that it's not the teacher's fault, that they can't do anything about it. I know that's not entirely what your intent with your post was, but focusing on external environmental issues and bemoaning the poverty of the student body can come across that way.
While poverty is a major issue, and teaching is hard, I don't think you can solve the problems by just changing the student's home environment. You have to change the teachers as well.
Lots of jobs are hard. Doctors have to deal with unruly patients who refuse to change their diet or exercise. Don't they get blamed when their patients don't improve?
How can society as a whole ensure that every child gets the best education possible?
Step one, define "best education possible".
Step two, learn to measure whether or not a child is getting one (or the closest you can come to it).
Step three, maximize some aggregate or regionalized version of your measurements through restructuring of school systems, rewards for good schools and teachers, etc.
The study in this article is just gets us to step two. It looks like you're wanting to discuss step three. That's a different can of worms.
It is not even clear if the article gets to step one. Some of the teachers who I feel had the greatest impact on my education taught classes that I did not necessarily score well in. If I am a statistical norm, those teachers would be consider poor teachers according to the metrics, contrary to reality.
This is where I have an issue with this discussion. You're throwing up anecdotal data in the face of an actual study. Why is that okay in a community like this that understands the Scientific Method and should have a basic understanding of statistics.
The only point I was trying to make is that test scores do not necessarily define the success of a teacher. I suggest, to be able to apply scientific method and get meaningful results will require far more data than is seemingly being collected.
What about prenatal counselling and early childhood nutrition programs and parenting classes? What if those things in underprivileged areas has more affect on educational outcomes (however defined) than teacher selection?
You may not control the home life of your engineers, but you presumably have at least some control over who you hire and fire. Even if you don't have the authority to fire someone directly you can still give your subordinates poor evaluations.
Imagine being a situation where not only did you not have any control over which engineers worked on your team - or even whether they were engineers at all, or even basically literate. Now imagine that your whole industry pretty much worked liked that and that changing from one company to another (which most likely worked the same way) always meant moving cities.
Incidentally, what they did in this study is to measure the value-addition of each teacher, not their raw test scores. Presumably you're not evaluated based on things you have absolutely no control over, as you said - you get judged based on the slice of the pie that you do control.
You make a valid point, and from the perspective of a teacher, or someone in charge of teachers, it is necessary to take your view. However, from public policy perspective, focusing strictly on teachers may be missing the mark.
As an aside, it occurs to me that those teachers who are most effective in underprivileged schools may be totally different than those in wealthier schools.
I think relativism does play a role on outcomes regardless of the parallels you are drawing here. However, you do make a point on a person's competence based on how well they can handle a situation but I don't think you can group one's role as a teacher being equal as one's role as a lead engineer.
I don't think you can group one's role as a teacher being equal as one's role as a lead engineer.
Why not? Government run education is pathetic. Incentives are all out of whack, there's no competition between teachers or schools that can affect the status quo.
Teachers could use a huge dose of whatever it is that the more competitive business world has to deal with.
Schools are the entity least run like a business out of anything else I can think of -- this is at the administrative level. When it comes to teacher competition, it is a different game than Company A winning a competitive contract over Company B. We want both companies to win here (read: both teachers -- or schools). Another problem with this idea is that you can fire obstinate clients. You can't do that with students.
Teacher competition, I think, would reduce collaboration. Why give your competitor your secrets to success?
Incentive pay? The problem under this idea is that new teachers are often given the most difficult and unruly of students while more experienced teachers get honors classes. I find this odd because the teachers will the best "tools" for dealing with difficult problems are moved away from those problems.
Then again, the problem is that with any idea for reform, there is a list of reasons why it should not or would not work.
I'm a fan of letting students who are not interested in the academics of school work towards trade school and let those who are academically inclined to work towards university. However, a retired college physics professor friend of mine does not like this solution because he would have been tracked towards trade school and never discovered his love for physics.
I recommend that you watch "Waiting for Superman". It focuses on the charter school initiatives across the country and shows that maybe we should be running schools much more like businesses. When you see how poor the options are for many of these inner city kids outside of the limited charter school options - it's heartbreaking.
Just like you'll see recommended on HN, if you're going to fail, fail early and get on with something that has a chance of working. Our government funded education system is a long painful failure that established forces prevent from being scrapped and reinvented.
I don't get to control the projects sent my way. I'm not allowed to "fire" customers with difficult expectations.
Hell, I've only ever fired one engineer in a 20+ year career, and that was because he was the equivalent of a violent student whom you would expel and send to juvi.
Why is it so hard to deal with the information as presented in the study. Fine, question the details or methods of the study. Offer counter evidence. The study presented evidence that good teachers can have a noticeable and measurable level of control on the output of their students. Why is dealing with evidence not the central part of this discussion?
I'm not sure why you're talking about "firing" customers. That might be the equivalent of teachers firing student's parents? Anyways, what I was trying to get at is that people working at a company choose to work there, MOST of the time they are doing something they are good at. Compare this to students, teachers do not get to choose their students from a set of applicants. Also oftentimes students do not see the benefit of being in school. I am not opposed to the study, I was only addressing the point you made about comparing a school to your company. I do not think it is a very good comparison.
> How is it any different than the rest of the working world where you can control some things and you can't control others yet your job is to do the best you can with what you can control?
So the argument is that since everyone sucks at measuring things that any arguments about how a particular system of measurement is ineffective should be discarded?
Sorry, I wasn't trying to create a straw man, just trying to tease out a hidden premise, although I suppose I could have been less of a dick about it.
I think we can all agree that judging people based on things that they have no control over is not an effective way to evaluate them. This happens in education, as the parent comment outlined. This happens in engineering, as you outlined. It does not follow that teachers should be evaluated in this way, or that a discussion about the problems with this method should not take place.
I really appreciate the way you're saying it, but I still think you're reading things into my words ("everyone sucks at measuring things" was not even a hidden premise). Worse, you're not acknowledging the evidence mentioned in the study.
The study attempted to control for things the teacher's couldn't control. They're actually being differentially judged on the parts of education that they can control.
These lines from the article fly in the face of your entire argument: "They also ranked the teachers using a statistical model known as value-added modeling, which calculates how much an educator has helped students learn based on their academic performance over time".
You would need to address them directly in order to mount a credible counter-argument.
So, if a teacher has as little effect on a child's performance as you seem to be implying should we then reduce teacher pay, accept the lower quality applicants we would get since it doesn't matter, and spend the extra money on anti-poverty measures?
So true! All of those factors exert an enormous effect on the outcome of a student's education. Unfortunately (except for the school resources issue) they are all completely out of the teacher/school/district's ability to control.
Whenever teacher/school issues come up, it is important to remember that far beyond anything a school or teacher can do, the single-most important factor in a child's education (or their life in general!) is their PARENTS. If a kid has parents who value education, discipline their kids and make time for learning at home, that kid can do just fine no matter what kind of conditions they experience at school. Conversely, it takes a Stand and Deliver-style superteacher to be able to coax success out of a kid growing up in a home where education is not valued and productive personal habits are not modeled.
The biggest problem with education? There are far, far too many parents out there who have absolutely no business raising children at all. We require tests for people to operate a jet ski, but when it comes to absolute power and responsibility for a human child, anybody can do it! That's your right! Don't worry about saddling the rest of society with your child who doesn't stand a snowball's chance in hell of ending up well-adjusted, educated or successful. It's YOUR kid.
I understand that people who are struggling with extreme poverty are often too caught up with bare survival to find the time to focus on their children's educational needs. That is really awful and sad. It is especially sad for their children.
The fact is, as long as society is full of self-centered, thoughtless parents who either fail to understand the basic components of child-rearing or are too busy surviving to care, we will be plagued with students who are going to struggle no matter how great of an educational system they are placed in.
That being said, perhaps the only way to raise a generation of education-valuing, quality parents is to do what we can to create them. So maybe it is best after all to focus on what CAN be done - effective teaching methods for today's teachers. Hopefully a much-improved school system can help raise the prospects of children whose parents are doing nothing to give them a hand up. And then they in turn can become the kinds of nurturing, education-valuing parents that their own children deserve.
That's why they do value-added testing rather than just evaluating teachers based on the score of their pupils. Obviously there are confounding factors, but if a teacher can make a statistically significant difference to the the test scores - as measured by before and after testing then that is valuable to know.
Irrelevant. The study uses Value Added Modeling, which accounts for these factors.
Unless the student's parents started abusing them in the year the teacher is being evaluated, it is already accounted for. VAM measures deltas, not absolute levels.
Namely, in the scatterplot correlating predicted effectiveness and actual effectiveness, they pooled teachers into 5% groups. This naturally reduces the variance, yielding a much straighter line of points than would an ordinary scatter plot. This is not, to my knowledge, an accepted or ordinary technique.
Ugh, right from the get-go, Rubinstein throws away any attempt at statistical rigor by using an outlier example to try to prove something.
The 5% groupings doesn't evaluate the metric. It just means that if they plan to use this actual study as a guideline, they should use the same technique in grouping teachers.
I don't know enough about statistics to judge the strength of Mr Rubinstein's critique, but I find this a little odd: The axes on his graph is a 'raw score' between -1 and 1, but the axes on the MET study graph is 'standard deviations'. Is that significant?
Actually they have figured out how to evaluate teachers based on statistical models based on their repeatability of success. Finding a 'great' teacher is much more than finding the top rated teacher in my opinion. What qualities makes such a teacher good is a totally different matter. The latter was what I was hoping to find.
"We’ve figured out what makes a good teacher" They've figured out that a "good" teacher can repeat the process of success (whatever this is), there is no information how he made it. The headline is slightly misleading.
The good news is they have video of good teachers (and bad) at their craft for years. It will take more analysis, but they have all the data at the ready.
They haven't figured out a damn thing regarding what makes a good teacher. At best, they've figured out how to find which teachers are good. That's not nearly as helpful. It's barely helpful.
What happens when you find the bad teachers? Fire them and hope the fresh-faced college grads do a better job?
So much of this debate on teachers is leans towards the carrot or the stick. Wanting to reward the best teachers because it makes you feel like you're helping is fine, but don't fool yourself into thinking it'll hugely improve instruction. Teachers aren't going to suddenly go, "Oh, wait, you'll pay me more if I'm a good teacher? I'm going to start trying!" (One or two might, but those outliers exist in any field, and are hardly the issue.)
In every district I've taught, very little time is given for teachers of the same subject to collaborate, observe each other, and improve their own teaching. Every year, we have to get 20 hours of professional development. Tell me, are 6 minutes and 20 seconds of development per teaching day going to improve instruction?
at which the latest project report can be found. I've got some reading to do before I think about what else to add to the discussion already underway here. I get the impression that one observation that has motivated much of this research is how persons educated overseas have fared in United States society after immigrating here in early adulthood. My own experience living in another country (twice, in two different three-year stays) does much to prompt my interest in education reform, which is what drew me here to Hacker News.
You can find a measure that correlates to performance, but once you use it to actually evaluate teacher, teacher will adapt and the measure will lost all relevance.
Rely on test scores? Teach the test, not the material.
Rely on student evaluations? Bribe them with easy material and good grades.
Rely on principal evaluation? Get on its good side.
This work is valuable, but I personally would like to put more focus on improving teaching and learning - helping teachers do better, helping students do better, creating better learning environments and curricula.
For example the budget of this study came out to about $15000 per teacher. What if instead we spent $5000 of that for professional development / training for the teacher, $5000 for a classroom set of chromebooks, and gave the teacher a $5000 raise. That alone would cause some significant improvements, I believe (not that that is ALL that is needed, and there are so many bigger issues that need to be tackled as someone else mentioned, such as poverty and inequity).
But when we talk about "good" teaching and "bad" teaching, we are treating it like it is a simple craft that is mastered or not mastered. Teaching is like engineering, it involves design, and design is about making things better (see this prezi on learning design: http://prezi.com/b44jwdgvs8nl/olds-mooc-introduction/ )
Because this was a study, the money spent has little to do with the individual subjects. If pollster calls me for a a national opinion poll I would rather be paid 1/1000 of whatever the budget was than answer the questions, but obviously my participation is more helpful overall.
The goal was to identify what actually works, which is something you need to know before improving the situation. Paying a bad teacher more will not make the education provided any better (despite whatever the teacher's union says).
The study identifies teachers who are effective against a particular metric. Most identified are likely members of the subset of good teachers. Yet, the three R's as measured by standardized test outcomes are not the only measure of effective education, just one that is cheap to measure.
Dewey believed that the goal of education should be to produce good citizens. Systems with metrics based upon the model of student as consumer are more likely to produce outcomes biased toward producing consumers.
The entire set of good teachers includes those who inspire a lifetime of learning. It includes those who teach kindness, sportsmanship, and good humor. It includes those who teach subjects for which there are few students and in which standardized tests make little sense such as art and shop.
Maybe the study can point us in the right direction. But few districts will bring outside evaluators from Cambridge's famed institutions. Few districts will fund ETS analysis or purchase their tools.
What we are likely to get are processes localized to the political reality on the ground.
Evaluating teachers is all well and good, but the key to improving our educational system is not that. What is needed is improving teachers. See "Building a Better Teacher"[0]
Is this the same gates foundation that I saw on 60 minutes pumping GMO seeds and fossil fuel fertilizer to third world countries to "help" them out of their food crisis?
In fairness, I think I heard they realized something like "what works in america doesn't work everywhere" (bit of a cop out to save face) and moved toward more sustainable organic methods, but I'm not completely sure.
All I know is, if its that hard for them to figure out the most practical way to grow a plant, we should probably take their advice on growing the minds of our children with a grain of salt.
It is opinion either way but in my opinion not only isn't it necessary, but it clearly threatens global biodiversity. Proper sustainable farming techniques produce an abundance and a variety of food, so the extra 1% increase on a huge monocropped field isn't even something that makes sense for third world communities.
On the biodiversity thing, viewing your website leads me to believe you appreciate nature, so I'm just scratching my head that you haven't made the connection. Ohh well, downvote me all you want buddy. You are entitled to your opinion also.
If I'm reading the linked page correctly, these metrics were verified by correlating them with individual student improvement on a variety of standardized tests and "more cognitively challenging assignments in math and english". This is better than pure circular logic, but until those metrics are validated, these are suspect. In particular, everything seems to test for shallow understanding.
The title of this article, and perhaps the researchers' conclusions from their data, is wrong. They didn't find the criteria of a great teacher (that is, what makes a good teacher). They just found some correlations among the _qualities_ of good teachers. It's the total opposite meaning. They found results of good teachers, not causes.
I believe that studies like this are well-intentioned in that those conducting the research are truly trying to improve the state of education in the US. However, the result of their work I'm afraid is the 'quantification' of teaching as a profession.
So where part of the study showed that with randomized redistributions of students, certain teachers were able reproduce higher test scores -- your countering evidence shows otherwise.
Great. Where's your evidence?
My own evidence (experience going through school and generalization of how ability works in most professions) tells me that the good teachers in my life helped me to learn the information better and more deeply. My memories of those classrooms are of the subject matter and my desire to achieve in that teacher's environment. The crappy ones were noticeably so in that my memories of those classrooms was of misbehaving with my friends, having no desire to help the teacher teach me, and cramming to learn material for tests because I hadn't absorbed anything beforehand.
I just realized that I keep replying to you, crusso (lol). Cheers. I'm more interested in the demographics of the student population than the 'random distribution (1)' of students in that population to different teachers. What I mean by this is that I would expect 'good' teachers to be able to demonstrate their abilities with with relatively homogeneous groupings of students where those students are from relatively decent neighborhoods. If, however, this study included teachers in inner city schools with huge variance in the 'quality' of student that a teacher would receive at often vastly different levels of understanding in the teacher's discipline area, then I think the results would be interesting insofar as to allow others to dig into what makes these teachers effective.
1: If we are looking at a teacher who teaches a course with a pre-requisite, the grouping of students becomes more similar (ie, a trigonometry teacher is likely to have students both motivated to do well and to have passed the requisite lower level math classes, and their success is less surprising to me).
Cheers... I'm over-posting to this thread. I feel fairly passionately about education. It's one of those things that I feel is grossly unjust to kids of all socioeconomic and aptitude levels. We spend so much money on education but entrenched interests use their louder megaphones to hold kids hostage.
A study comes out from the Gates Foundation that tries to tackle some of the education problems head on. It produces some statistically interesting results, but here on HN, I keep reading knee-jerk responses about "standardized testing" from people who apparently didn't even read the attached article.
How are we going to solve these problems if no one even bothers to understand our options?
Regarding your point - I agree. Any solution would need to account differences in what you referenced as a "variance in 'quality' of student".
kudos to the WaPo for giving a link to the study. too often, many publications discuss a study but never link to it, which then makes it harder to go and review it for yourself.
Brilliant. Well, yes, that is the best way to measure a teacher's efficacy, if your goal is to teach students to jump through hoops and pass standardised tests rather than to think.
The best teacher I ever had was my grandmother - she taught me the most important lessons I'd ever learn, such as:
"Observe the world around you."
"Reach your own conclusions."
"What was fact yesterday may be fiction tomorrow."
I don't really understand the visceral reaction against testing in this HN thread.
You're tested throughout your working life. Almost every job interview is a test. Giving a customer presentation is a test. Finding bugs in your software for a release date is a test. Memorizing that big Cheesecake Factory menu so you can wait tables is a test. Getting your General Contractor's license is a test. Getting certified as a Pilot requires passing a test.
There should be more to education besides testing, but being tested is a part of life and it's easy to measure success - that combination means that it's a useful tool for education.
Job interviews are a good example of a kind of testing that we know doesn't work all that well but we keep around for lack of a better alternative.
The others are not good examples because they are useful in-and-of themselves. Giving good presentations and finding bugs are useful because they are useful, not because they predict something else.
Pilot's licenses, like driver's licenses are based on tests where you more or less do the thing that you're being tested on. The test and the thing that it's supposed to measure are very similar and we can therefore use it to predict performance. In cases like that "teaching to the test" doesn't exist because practicing for the activity and for the test is the same thing.
I'm not denying that educational testing is useful to quantify progress but such tests are often different enough from the underlying activity they're intended to measure that effort spent preparing for the test has little other purpose.
Job interviews are horseshit. I've been on both sides of the table, usually the hiring side. I've encountered people who excel at interview, and turn out to be useless. Conversely, I've encountered people who are horrible in interview, and wonderful employees.
Giving a presentation isn't a test, it's a pleasure. You're not being tested. You're just presenting facts for someone to make a decision. There's not a right or wrong. If there is, you're in the wrong room.
QA is not a test. It consists of tests, sure, but those are for the software, not you. It's a task, and one with a purpose.
Memorising a menu is not a test. It's memorising a menu, for a purpose. If you were memorising a chapter of Crime and Punishment verbatim to get a job at Cheesecake Factory, that's a test, and an arbitrary one.
Getting a license, in the sense that you're tested, again, is different, as those are vocational tests which test your practical skills for a specific application.
Being tested is only part of life if you choose to let other people set your challenges. This is what the educational teaches you to do, and accept as normal.
In middle school, the last three weeks of each semester were spent watching movies (the same two or three movies over and over). In high school, I was taught that the big bang theory was when the continents divided. I was taught that the most direct route between two points on the globe is through the north pole. My Spanish and woodworking teachers taught math because they knew that the math teachers weren't doing it.
When you're talking about schools like the ones I went to (which weren't nearly as bad as many other schools in America) I think that test scores are a decent indicator of the education a student is receiving. You're measuring things like "can this kid read English" and "can they multiply two numbers together." Standardized tests are certainly capable of measuring things like that. Testing probably breaks down for measuring intelligence/education as the level of education improves, but I encourage everyone to consider just how dysfunctional many schools are before ignoring testing entirely.