One Step At a Time

Rating schools under the No Child Left Behind Act has produced mixed results. Now it's time to grade the law itself.

Joel Klein likes the 2002 No Child Left Behind Act.

The chancellor of the New York City Department of Education, the nation's largest school district, believes in the central mission of the federal law: to improve the academic achievement of the country's children, especially those who are usually left behind-poor, minority and special needs kids.

As someone who thinks deeply about management and leadership issues, Klein likes the fact that through NCLB, the federal government measures schools' progress toward a basic goal: getting American students reading and doing math at grade level by 2014. Klein has been both a federal and private sector executive in fields other than education, so he recognizes that what gets measured gets done.

The national reading and math goal is daunting. Only about a third of fourth- and eighth-graders performed at grade level on the 2005 federal test in reading and math. In New York City, the numbers are even lower, ranging from 16 percent of eighth-graders proficient in math to 23 percent of fourth-graders proficient in math. Just 18 percent of New York City fourth- and eighth-graders are proficient in reading.

So Klein has his work cut out for him. His school district has about 1.1 million students in 1,400 schools. More than 80 percent are eligible for free and reduced-cost lunches, and poor students tend to arrive on schools' front steps with a lot of problems that make it hard for them to do well academically. Still, Klein thinks NCLB accountability can focus principals and teachers on getting disadvantaged students in shape for academic success.

At the same time, he is worried that the accountability system is not quite right. It has the right goal, but measures the wrong things. He worries about the perverse incentives the current system might create-a concern anyone who has ever developed performance measures can understand.

To illustrate his concern, Klein tells a story. He recently visited a school where performance had gone up 20 points over the previous year, with an increasing number of students scoring at grade level-at Level 3 of the four-level state tests in reading and math. NCLB uses those tests to assess schools. Klein said to the principal, "You had such great results. You went up 20 points last year. How'd you do it?" The principal said he identified the students who were just above Level 3 at the start of the school year and worked with the teachers to make sure they didn't fall back. He also identified those just below Level 3 and worked with their teachers to move them up to Level 3.

"What happened to the other kids?" Klein asked. "They probably did OK," the principal answered.

Klein had discovered a strategy principals were adopting across the country: Focus on the kids just below and just above the cutoff scores on reading and math tests-the "bubble kids," as they are known-not on the other students. "It's a rational strategy, if you believe accountability leads people to count what counts," Klein says. But the bubble kid approach ignores children who perform far below grade level-those who need the most help-and those far above it-who still should be expected to improve. That's not what the crafters of the No Child Left Behind law had in mind. "I wouldn't discount the power of what we measure in terms of the behaviors in a school system like mine," Klein warned legislators at a hearing on the law this summer.

Audacious Experiment

No Child Left Behind is perhaps the largest performance measurement experiment in the history of the nation. It is the first nationwide accountability effort for the public school system, which encompasses 50 million students taught by 3 million teachers in 90,000 schools in 14,500 school districts in 50 states. Each state created standards in reading and math for every grade level and annual tests to determine whether students were meeting those standards. They created or modified data systems to analyze those tests, school by school, and issue report cards. Four years into implementation of the accountability system, educators are starting to draw lessons about how it is driving change, for better or worse, in the nation's schools.

To appreciate the audacity of the central No Child Left Behind goal-getting all kids reading and doing math at grade level just eight years from now-consider some of the facts confronting the nation's educators. Two-thirds of students currently aren't performing at grade level in reading and math. About a third of students are eligible for free or reduced-rate lunches, a rough indicator of how many students come from low-income homes. In big city districts such as New York, a majority of students come from poor families. About 13 percent of U.S. students-roughly 6 million-have individualized education plans, an indicator that they have special needs. Such special needs can range from dyslexia to severe cognitive and physical disabilities. Nearly 11 percent-about 4 million students-are English language learners, meaning their first language is a foreign tongue.

In 1965, the government recognized poor students were failing and started investing billions of dollars a year through the federal Elementary and Secondary Education Act to provide extra help to schools serving them. But 35 years of federal involvement didn't close the performance gap between wealthy, usually white, students, and poor, mostly minority, students. As the 20th century closed, political leaders decided the Elementary and Secondary Education Act wasn't getting the results they wanted.

Republican President George Bush and Democratic Sen. Ted Kennedy of Massachusetts, among others, worked together in 2001 to add a requirement to the federal education aid: If states wanted money, they had to show results. The No Child Left Behind Act set the 2014 goal, and required states to set incremental goals for each year between 2002 and 2014.

Each year, states tabulate reading and math test results and issue report cards for every school. The report cards show whether a school met all the goals for that year. For example, this year, the incremental goal in Pennsylvania is that 54 percent of students must be proficient in reading and 45 percent proficient in math. The report card tells the school's principal and teachers how many students overall were proficient in each subject. The report card also breaks out the results by subgroups of students-white, black, Hispanic, Asian, American Indian, multiethnic, special education, English language learner and economically disadvantaged-if there are at least 40 students of that type in the school.

If a school meets the targets for all students and for each subgroup, then the report card says the school made "adequate yearly progress." If the school fails to meet the target for any one group in any one subject, then it did not make adequate yearly progress.

It's a pass/fail system. If a school passes, nothing happens, although some principals celebrate with ice cream parties or pep rallies if they make adequate yearly progress. If a school doesn't make adequate progress, it gets a warning and the principal is encouraged to figure out a way to make the grade the following year. If the school fails again, it must offer its students the opportunity to transfer to another public school and pay for their transportation. If it fails a third time, the school must provide tutoring, either through the school district or through a private education company. If it continues to fail, the school district can hire a consultant, develop a detailed plan to improve performance, fire the principal, dismiss teachers or shut down and reopen as a charter school.

Do You Believe?

Raymond Simon is the deputy secretary of the Education Department. One of his duties is to oversee the efforts of Klein and other state and local education administrators as they try to meet No Child Left Behind goals. Simon has been chief state school officer for Arkansas, superintendent of the Conway, Ark., School District and a high school math teacher.

He acknowledges that some principals have responded to No Child Left Behind by focusing on bubble kids. But, he says, many have not. "You don't have to focus on these children and exclude everyone else," he says. Simon visits schools throughout the country that have improved the performance of students at every level, and says they all have two common attributes.

"They're pretty consistent," he says. "The first thing they do is they believe that kids can do it." Simon contends that the 2014 goal for getting all kids to grade level is realistic and doable-a line of reasoning that is controversial among educators. Many teachers say it is unrealistic to expect schools to erase the effects of poverty and weak parenting that keep students from doing better. But according to Simon, teachers at schools that are improving reject those arguments. "2014 is today in more and more of our schools," he says.

Second, educators in successful schools "use data to drive instruction," he says. By that, Simon means they regularly assess which reading and math skills students lack and spend time helping develop them. Principals who see common skill deficits urge teachers to try different instruction methods. It's "testing for teaching, not testing for accountability," he says.

Simon's two discoveries are not surprising, since they are two elements of any successful accountability system. People must agree with the goals and believe the measurements are valid. Otherwise, they ignore or game them. And people must use the accountability system as a tool to highlight what needs fixing in order to improve results. No Child's proponents say it is having a positive effect:

• Achievement levels have gone up. Younger students in particular have done better on reading and math tests, with scores for all students, and for black and Hispanic students in particular, increasing on federal tests between 1999 and 2004. Education Secretary Margaret Spellings, who helped craft No Child Left Behind, says there's been more improvement in the last five years than in the previous 28. The Council of Great City Schools, an urban education group in Washington, reported earlier this year that the percentage of urban students working at grade level went up between 2002 and 2005.
• Educators are talking about results. At every level-federal, state, school district and schoolhouse-educators are discussing adequate yearly performance requirements and trying to figure out how to get students to do better and low-performing schools to make the grade.
• Schools are focusing on poor and minority students. Because the accountability system reports on minority and poor students' achievement-as well as special education and English language learning students-schools are paying more attention to them.

But even many supporters of the law, such as Klein, say the system is flawed. Teacher unions and many education lobbying groups, some of which were opposed to the law as it was written, see additional problems:

• It is too blunt. Because it is a pass/fail system, schools that are struggling with only a few special education students are lumped in as failures with systemically poor performing schools. A more refined system would help school districts target assistance-or punishment-more effectively. Klein argues for a graded system-A, B, C, D and F-to give educators and parents a more "granular" understanding of schools' struggles and achievements.
• Progress doesn't count enough. Schools make adequate yearly progress if enough students move from below basic or just basic achievement to grade-level performance. That approach misses on two counts, critics say.
  
  One, as Klein's story about the principal illustrated, is that no credit is given for getting more students to advanced levels or for getting more students from below basic to basic performance.
  
  Second, the system measures progress by comparing the scores of students in one grade to those of students in the same grade the next year. Critics would measure the progress of one group of students as they move up. One third-grade class might have more advanced students than the next year's class, making the annual progress review an apples-to-oranges comparison.
• Teachers and principals haven't bought into it. Whether or not it's a good system, administrators say teachers and principals have trouble accepting it. Some believe it's merely a way for politicians to label public schools as failing.

Others think it is forcing them to teach to the test, as many teachers say, and narrowing the curriculum to focus on reading and math at the expense of other subjects. Still others note the system is punitive and doesn't reward schools that make their numbers. "I need to be able to convince those people the measures we use are really very powerful and fair," Klein says.

Tweaking the System

The No Child Left Behind Act is up for renewal in 2007. At that time, lawmakers might consider changing its measures. But some, such as Rep. George Miller, D-Calif., one of the law's strongest supporters, worry that focusing on adjusting the measures would draw attention away from improving student achievement. "It's not just about changing the standard," Miller says.

Klein says the system needs to be adjusted to encourage improvement for all students: "What you really want to see is a kinetic motion of all the kids moving." Simon agrees in principle, though he contends many schools already are seeing that kind of movement.

The Education Department has granted flexibility to states and school districts to handle their concerns while maintaining the integrity of the law. "This whole journey is going to be a matter of adjustment," he says.Regardless of what happens next, the No Child Left Behind accountability system has drawn attention to what educators are doing to improve student achievement and to whether they are succeeding.

Klein knows people are watching. "What we are doing in New York is more than a test of whether we can reform our schools," he said in a recent speech. "It's a test of our ability to make government work."