Report: States gaming NCLB system

(Updated Thursday, Nov. 15, 5:15 p.m. EST)

On paper, Alabama last year showed remarkable gains in improving its schools. But a new report claims that Alabama – and a number of states – are manipulating statistics to make their schools appear better than they really are.

 

The report released Tuesday (Nov. 13) by Education Sector, a nonpartisan think tank in Washington, D.C., contends states are gaming the system under the federal No Child Left Behind Act (NCLB), the 2002 law that measures states’ annual progress toward getting all students reading and doing math at grade level by 2014.

 

In a ranking based on 11 statistics that states annually report to the U.S. Education Department, Alabama jumped to 5^th place in the country in how well it appears to be meeting various education measures, up from 22^nd place last year, the group found. The percentage of its schools showing the amount of improvement required by federal law zoomed in one year from half to almost 90 percent, according to the report.

 

“This didn’t happen because Alabama students learned much more in 2006 than they did in 2005,” the report said. It happened because the state exploited loopholes in the law and set low standards for its statewide test so that more students passed those tests, inflating the state’s record in meeting the law’s benchmarks, according to the report.

 

“Many states did exactly the same thing,” said the group, which contends Congress needs to close these loopholes when lawmakers rewrite NCLB this year or next.

 

Independent measures peg Alabama as having one of the country’s lowest high school graduation rates and some of the lowest scores on the National Assessment of Educational Progress (NAEP), a test taken by a cross-section of students in each state.

Alabama officials dispute the report’s analysis of their performance. Dr. Gloria Turner, the state’s director of assessment and accountability, said that over the past decade Alabama has put more resources into education, such as a reading initiative for K-3 students, and that such programs have resulted in improvements for the state. She pointed out that the state’s ACT scores have risen steadily from 2003 to 2007, and that the 2007 NAEP results showed Alabama had made the most improvement of all the states in fourth-grade reading scores over the last two years.

“We have really taken NCLB seriously,” Turner said. “When you have sanctions out there that are attached to something like NCLB, and you put into place educational programs to help improve that – we would be extremely disappointed if we weren’t making improvements.”

The report went on to highlight Iowa and Wisconsin as states that perceive their educational systems are doing well, even though report author Kevin Carey contends that Wisconsin also has manipulated the numbers to present the state as doing better than it really is.

“Wisconsin, for the past two years, has adopted every means of not holding schools accountable for No Child Left Behind,” Carey said.

 

The report, “The Evidence Suggests Otherwise,” creates an index based on 11 data measures that states submitted to the U.S. Department of Education, such as high school graduation rates, high school dropout rates, number of violent schools, and the percentage of students, schools and districts performing well under NCLB.

 

Iowa and Wisconsin tied for first in this year’s index, similar to their ranking in last year’s index.  

 

Alabama stood out as the state that appeared to make the biggest improvement. But the report detailed some ways over the past five years that the state inflated its progress, such as limiting the number of test scores used to chart progress in closing the achievement gap between minority and white students. Alabama chose to single out test results for a subgroup only if a school had 40 or more students in that group. In the 2002-03 school year, the year states chose where to set their minimum subgroup size, 80 percent of Alabama’s schools had Latino students, but only 8.7 percent of schools reported test results for Latino students as a group.

 

The higher a minimum subgroup size, the more minority and disabled students are excluded. More than half the states set subgroup cutoffs lower than 40 so they can include higher percentages of those scores, while 11 states chose to share Alabama’s strategy of counting scores of minority and disabled students only in schools with 40 or more such students. Seven states set even higher cutoffs.

 

Alabama’s neighbor Tennessee pioneered another loophole. Originally under NCLB, states averaged test results across multiple grades in a school district to determine whether districts made a target, or whether they would be labeled “in need of improvement” for missing it.

But in 2003-04, Tennessee received permission from the U.S. Department of Education to set a lower standard: the state would not have to label a district as in need of improvement as long as the district made its target in at least one subject in just one of the three grade-spans: elementary, middle or high school.

 

In the next two years, 28 states including Alabama followed Tennessee’s example. This helped Alabama go from having no districts making annual progress one year to having 63 percent of districts make progress the next.

 

Another tactic states use is known as the “performance index.” That means states could get half credit for students who weren’t proficient, but who still reached a basic level. In 2005-06 Alabama borrowed this move. Since the state already had up to 85 percent of students at a basic level, almost half the non-proficient students now counted as proficient.

The report also criticized the Department of Education for approving states’ requests to make it easier to make annual progress. A spokesman from the department said that for the first five years of NCLB, the department’s focus was to get the core testing structure in place, improve failing schools, set up a system to improve data collection, and share that information with the public.

“It is largely because of NCLB’s testing and the data collection requirements that this report can even be written,” spokesman Chad Colby wrote in an e-mail.

The report contrasts the tactics of Alabama, Tennessee and Wisconsin with states such as Massachusetts. Although the Bay State’s students are the top performers in the country on NAEP, Massachusetts ranked 46^th in how well it considers its progress in improving its education system. Massachusetts “continues to hold itself to far tougher standards than most” states, according to the report.

Alabama’s Turner questioned the report’s methodology, saying that out of the 11 data measures, only about half have to do with making adequate yearly progress under NCLB, yet most of the study “is about how Alabama and the other states and the Department of Education have been in cahoots to usurp NCLB.”

Editor’s Note: This article was updated to correct a reporting error and delete a reference to Wisconsin as the first state to use the “performance index” tactic. While Wisconsin uses the tactic, it wasn’t the first.

Comment on this story in the space below by registering with Stateline.org, or e-mail your feedback to our Letters to the editor section at letters@stateline.org.

Contact Pauline Vu at pvu@stateline.org.

See related stories: 
Lake Wobegone, U.S.A. -- Where all the children are above average
Govs call for more control over NCLB
No Child law faces medley of changes