Unfortunately gaming the system is still a temptation.
By rewarding growth, schools are incentivized to prevent earlier grades from doing too well.
One student with potential pushing the overall performance higher than that school might like for that year might be purposefully stunted in order to allow room for growth in later years, robbing them of getting as far in life as they would have.
Do we think this is likely? Surely if we also have an incentive that compares schools to each other in terms of their test scores at the same time -- which we currently do now -- then schools could be both incentivized to a) maximize the scores at every grade, and b) increase the scores the most?
Since state standardized tests are now adaptive, there is no reason to have the student who comes in knowing everything at their grade level to appear stagnant in testing. They can test beyond their grade level. That would incentivize schools to do something for those students instead of to ignore them.
I agree gaming the system is always a temptation with accountability systems.
I’m not sure it would work like how you are proposing, since purposely stunting a student would penalize the school exactly as much as they would “gain” as the student catches back up.
As a teacher, I agree that we should grade schools on student growth.
Give the kids 3 standards referenced tests a year; one is referenced to the school district, one to the state, and one to the nation. Give all 3 exams in a week when they do nothing else. This gives us better test results than a single exam on a single day.
In the state of New York, I believe during COVID our state tests became optional. In order for your idea to work, standards referenced tests would need to be coordinated _and mandated_ across three levels of government (sometimes a 4th municipal because NYC has sway over most of their own schools granted by the governor).
Because the state tests are now optional they are less influential as inputs for future years (almost by definition), and/or alterantive pathways and rubrics must (and are) provided which further dilute their effectiveness.
It's a tricky and complicated situation, and obviously school districts, states and even the federal government are all trying out different ideas.
This idea sounds familiar. Isn't this basically "Value-added Modeling", which looks at how much teacher contribute to individual differences in increases in student test scores? [1] Has this not been applied at the school level!?
Surely growth measures require repeated testing (so one can see how the differences between student at Time A vs. Time B), not necessarily adaptive tests? Maybe I'm missing something; could you help me understand?
I'm abstract, you're correct, and a typical setup for an educational experiment is a calibrated pre / post test to see what changed over the course of interacting with an activity. Both tests are effectively identical and measure the same thing.
However, state tests are annual. You don't want to measure the same thing. If you don't measure the same thing, you can't compare.
There are technically non-adaptive test designs which might work, but it's not what we're using, and they're a lot more complex than adaptive.
Also, the SBAC test administered in California, Washington, and many others is now adaptive, but still schools are measured based on point-in-time aggregates.
I can elaborate a little bit but not a lot since that would be essay-length. I can simplify and explain roughly.
The short story is that most states assessments are designed almost as binary measures, to see if students are above or below a cutoff threshold set by Common Core State Standards. They're designed to measure schools, and in particular, to flag failing schools where kids aren't meeting standards.
They're very good at that, actually.
However, that's almost meaningless as a measure for kids well above or below standards, or unaligned to standards.
"Growth" is almost meaningless here. If I know one number is less than three and another less than four, I can't subtract them.
It's more mathematically fancy, but that's the jist. It gets even worse since measures constructs are highly multidimensional and different dimensions are measured each year. It's like subtracting apples from oranges.
It also encourages the wrong behavior. For kids behind, I'll get the best "growth" by discussing on grade level material and leaving gaps for what kids failed to learn before. I'll also do well to ignore my students who are ahead. Indeed, students who did week last year will inevitably hurt my "growth."
As a footnote, I would not call this a strong claim. Talk to a psychometrician and you'll see it's common knowledge.
If adaptive tests move beyond those few states, the problem goes away.
That isn't the question that matters. What matter is what do we do with that knowledge? Because right now, the answer isn't a whole heck of a lot.
Systematically comparing schools on any metric accomplishes nothing. And every year for decades we act like we just need to identify the good schools and replicate their success, with literally actual regard given to what makes schools successful in the first place: functioning administration, involved parents, good and stable teachers. After that, curriculum and resources.
And a functioning administration is generally what gets sacrified first in pursuit of some across the board improvement in some flavor of the week metric.
Teacher’s unions and their enablers who are politically aligned to them reject all forms of measurement and accountability. Even this alternative one of growth instead of snapshots will be attacked on flimsy grounds.
By rewarding growth, schools are incentivized to prevent earlier grades from doing too well.
One student with potential pushing the overall performance higher than that school might like for that year might be purposefully stunted in order to allow room for growth in later years, robbing them of getting as far in life as they would have.