11/2/09

Arne Duncan: A Flawed Proposition


From the National Journal Online debate about school turnarounds:

Richard Rothstein, Research Associate, Economic Policy Institute

It is an admirable goal to "turn around" low-performing schools. But before attempting this, we need to ensure that we have accurately identified which schools are low-performing. It would be tragic if we aggressively intervened in (or even closed) schools that were, in fact, better performers, while ignoring schools that were worse.

This is the fundamental flaw in Arne Duncan's proposal. We don't, in fact, have any good ways to identify low-performing schools, so any turnaround efforts are likely to include considerable misdirection.



Indeed, as I have written in a recent Policy Brief for the Economic Policy Institute, the assumption that we presently know how to identify low-performing schools is inconsistent with principles of educational accountability that Arne Duncan has himself articulated.

At present, the only tool being used to identify low performing schools is the percentage of students who pass (i.e., are deemed "proficient" on) state standardized tests of math and reading, required by the No Child Left Behind law.

Because these tests are so flawed, they cannot accurately identify high- or low-performing schools.

Here is what we know about these tests:

a) they narrow and distort the curriculum by giving schools with low test scores incentives to abandon other aspects of a well-rounded curriculum – the arts, science, history, social studies, health and physical education, character development. Survey data confirms that such narrowing is in fact taking place.

b) they create incentives to reduce instruction to the most basic skills, because states can reduce the cost of testing by eliminating open-ended questions that more easily assess critical thinking.

c) they spur teachers and schools to focus intensive instruction on students whose past performance indicates they are almost ready to pass the test, while paying less attention to students whose past performance indicates they will easily pass. A school following this strategy (and again, there is considerable evidence, both qualitative and statistical, that many are doing so) can have rising percentages of students proficient, while its average scores are stagnating or even declining.

d) their scores are reported by subgroups that are not comparable. For example, "low-income" students are defined as those who receive free or reduced-price lunches, but this category includes students whose families are very poor, and those whose families have incomes up to 185% of the poverty line.

e) their scores are themselves increasingly inflated, as successive test years reproduce similar questions (again, to avoid the expense of designing entirely new tests from year to year) and teachers learn to focus instruction on those basic skills likely to appear on the test, while giving less emphasis to those unlikely to appear. State test scores are almost universally climbing, while scores on the National Assessment of Educational Progress are relatively flat. This is a good indication of such inflation.

Not every school responds to the perverse incentives of No Child Left Behind in the same way. A school that sacrifices some gains on test scores of math and reading in order to give children access to a broader curriculum may falsely appear to be lower-performing than schools that do not make this choice. A school that sacrifices some gains on test scores of math and reading in order to balance instruction in basic math and reading skills with opportunities for critical quantitative and literary thinking may falsely appear to be lower-performing than schools that do not make this choice. A school that sacrifices some test gains on "percent proficient" in order to ensure instruction for children at all points in the achievement distribution may falsely appear to be lower-performing than schools that do not make this choice. Schools whose students come from extremely distressed families and communities may have lower test scores, but falsely appear to be lower-performing than schools that contribute less "value-added" but whose students come from stable low-income families. And a school that sacrifices some test score gains because its teachers devote more time to instruction, and less to test preparation and drill, may falsely appear to be lower-performing than schools that do not make this choice.

If we want to turn around low-performing schools, the first task should be to ensure we are identifying these schools accurately. Such identification requires much more than test scores. It requires expert human judgment, with qualified experts visiting schools to interpret test scores and evaluate the overall quality of instruction. I have described the need for such an accountability system in Grading Education and the Broader, Bolder Approach to Education campaign has proposed such an evaluation system to replace the flawed identifications produced by the testing mandates of No Child Left Behind. Without such an evaluation system, the Race to the Top ambition will, in retrospect, turn out to have been yet another blunder in our test-obsessed school accountability policy.

Total Pageviews