Tuesday, October 12, 2010

Too little and much too late, the Times finally reports on the state test score scandal

In yesterday’s front page story, entitled "On NY School Tests, Warning Signs Ignored," the NY Times' account of the state test score scandal left its own deficient reporting conveniently off the hook.

Anyone who was paying attention knew at least as far back as 2007 that there was rampant test score inflation, primarily through articles by Erin Einhorn and other reporters at the Daily News. These articles, which themselves relied on analyses from testing experts like Fred Smith, revealed that the test score inflation started as early as 2002, with questions and scoring on the state exams becoming easier over time.

See this 2007 article on our blog by Steve Koss, relating the ingenious experiment done by Einhorn in which she gave the 2002 and 2005 math tests to the same bunch of children, with the results showing that the 2005 exam was much simpler, a fact also reflected in the changing "P" values of the questions. Or this follow-up Einhorn article, where leading testing experts called for an independent audit, which of course did not occur until three years later.

Where was the NY Times amidst all these revelations? Absolutely nowhere. Even now, the Times article omits any mention of the Daily News’ earlier exposes – which brought attention to this issue to the wider public – and instead recounts as somehow meaningful that a few individuals who supposedly had doubts about the apparent rise in test scores, like Pedro Noguera and Kathleen Cashin, didn’t directly mention them to Klein– as though he might otherwise not have noticed the evidence that was splashed all over the Daily News!

The article also offers a rather irrelevant story, relating how Joel Klein earnestly tried to convince the state to change its scoring system to use a value-added method instead, as though that would have addressed any of the problems regarding the score inflation. In fact, that might have made things worse, as indicated by the way in which the inflation led to 84% of all NYC elementary and middle schools receiving "A" last year, based primarily on the value-added method, falling precipitously to only 25% this year, when the state decided to reset the cut scores.

The article also gives Regent Merryl Tisch a pass, letting her have the last word, saying “We came in here saying we have to stop lying to our kids,” without mentioning that throughout the test score inflation period, she was Deputy Chancellor of the Regents, and yet reliably supported Bloomberg and Klein's claims of great improvement.

The Times itself had plenty of reason to know about concerns about the state test score inflation throughout this period but not only failed to report on it, but generally toed the company line.

On August 4, 2009, at the very moment when Bloomberg was pressing for the extension of mayoral control of the schools, and two years following the Daily news exposes, the Times published a credulous story that recounted the steep increase in state test scores and the apparent narrowing of the achievement gap, including this quote from Joel Klein:

Mr. Klein, for his part, said he was confident that rising scores reflected real improvements. “No matter how you look at them,” he said, “the picture is one that shows that the city is making dramatic progress.”

In the article , Klein put more emphasis on the apparent rise in proficiency levels rather than scores, “saying that the pass-rate was the more critical measure because it indicated proficiency, an important gateway to success…Our job is to get all kids to basic proficiency and then continue to move them forward, and I think we can do that.

Now, of course, Klein emphasizes the higher scale scores on the state tests, since the reported proficiency levels have dropped so dramatically.

To the degree that the Aug. 2009 Times article expressed any doubts about NYC's dramatic rise in test scores, it was only the possibility that the drive towards high-stakes accountability had led to excessive test prep-- not any of the overwhelming evidence that the tests were simply and the scoring more lenient.

Indeed, yesterday's article focused on the same set of concerns -- excessive test prep, the public release of prior exams, and the limited number of questions on the exams -- rather than any of the more damning findings, showing how the state had rigged the results with easier questions as well as lower cut scores, both of which would directly implicate the state in the fraudulent outcome.

As I wrote in August 2009 to the Times editor, Ian Trontz:

“… there are many prominent administrators, researchers, teachers and principals who believe strongly that there has been rampant state test score inflation in recent years. Not only are huge jumps in the scores occurring in nearly all districts and in all grades– the middle grade increases last year were especially unprecedented -- but as has been widely reported in the Daily News and elsewhere, the cut scores have been lowered each year. To leave this out of your story seems negligent at best, especially given the room and the time your reporter had to expound on this issue.”

I also pointed out that the article had wrongly attributed gains to Bloomberg/Klein on the NAEPs by giving them credit for the increase in test scores since 2002, even though their reforms had not begun until 2003. Here is how Trontz responded to this point:

"We do not, however, think that your email points out any inaccuracies. It is not incorrect to say that fourth grade reading scores rose after the mayor took over. You are correct that some of the biggest gains occurred before his major reforms took effect, but we are not incorrect."

I concluded at the time:

Given the evident bias of this article, it appears clear that the Times has been captured by the Bloomberg/Klein PR machine, and can no longer be trusted to provide objective analysis of their education record."

Shortly afterward, Wayne Barrett wrote about the controversy in the Village Voice,

"The Times front page piece last week -- headlined "Gains on Tests in New York Schools Don't Silence Critics" -- failed to quote any real critics, but gave Klein six self-promoting paragraphs. It did bury a single questioning quote from two academics not known as critics of the test scores in the thirty-fourth paragraph, but the top of the story trumpeted success scores that would have silenced any critic. If, that is, they were true."

Two days after the NY Times article ran, on August 6, the NY State Senate voted to renew mayoral control without any checks and balances, essentially allowing Bloomberg to retain his stranglehold over our schools. One of the few Senators who voted against the renewal, Sen. Carl Kruger, argued that the achievement gains claimed by Bloomberg and Klein would soon be found to be fraudulent.

In response, the Daily News editorial page, as Bloomberg-sycophantic as all the other NYC major dailies, argued vociferously against Kruger's claims. In the piece, the News editors referenced the recent NY Times article (since they could not cite the far more explosive reporting of its own staff ):

“Good luck with that. He'll sooner find Sasquatch under Chancellor Joel Klein's desk. The New York Times double-checked test results and concluded this week that they showed "a steady march upward."

Indeed, Sasquatch was hiding in plain sight under Klein’s desk all along.

Just a few days after the Senate vote, on August 11, 2009, Meredith Kolodner of the Daily News pointed out that the cut scores had been set so low on the 2009 exams that a sixth grade student could pass the ELA just by randomly guessing, while 7th graders had to get just one extra question right to pass. At Gotham Schools, Diana Senechal tried taking the exams herself and confirmed that in the 5th grade ELA, 6th grade math, and 7th grade ELA exams, a student could indeed achieve a level 2 through random guessing .

What’s most scary is that according to the latest Times article, the state is now apparently going to keep the questions on the exams secret forever, supposedly to ensure that they can keep their difficulty level stable -- to guard against excessive test prep using old tests.

So in future, it will be impossible for another reporter like the intrepid Einhorn to test the same bunch of kids with exams from two separate years, to prove how much easier they have become. Or for any parent to examine how flawed the questions may be. (Remember Brownie the Cow, the absurd questions on the 2006 4th grade ELA exam?) As though keeping the questions secret is the only method that can be used to keep standardized exams stable over time!

Meanwhile, companies like the College Board manage to release the SATs each year, and still are able to equate them, by holding back the ungraded questions (what the Times article calls “field test questions”). Why that could not occur in the case of the NY State exams is never explained by the article.

Sorry, NY Times, this article is too little and too late. Years before, when Bloomberg was pushing to retain unilateral control over our schools based upon these inflated test scores, the paper of record owed it to its readers to inform them of these issues, and yet utterly failed.

(See my critique at the time of their August 2009 article, NY Times falls in line with the Bloomberg PR spin control; and the response from Times editor, Ian Trontz: The NY Times response, and my reply.)

1 comment:

salma hayek said...
This comment has been removed by a blog administrator.