@ Penn GSE: A Review of Research
Fall 2007

In Practice

Rethinking the Achievement Gap

By Andy Porter

Back in the 1960s, the noted sociologist Christopher Jencks called for income tax redistribution to address the issue of racial inequality. Today, he looks to education: “Reducing the test score gap is probably both necessary and sufficient for substantially reducing racial inequalities in education attainment and earnings.”¹

Jencks is not alone in this assessment. In the last 40 years, more has been written about the achievement gap than just about any other topic in education. But what exactly is the achievement gap? How important is it? What has been done, and what can be done, to address it?

What Is the Achievement Gap?
The achievement gap is the persistent disparity in academic achievement between minority and disadvantaged students and their white counterparts.  To begin my discussion of the issue, I feel I must in some way account for the nature-nurture tension that sometimes underpins conversations about the gap: suffice it to say that I weigh in with Richard Nisbitt, who stated that “[t]he most relevant studies provide no evidence of the genetic superiority of either race but strong evidence for substantial environmental contributions to the IQ gap between blacks and whites.”²

In my view, it is not innate ability but rather the opportunity to learn—an artifact of environment—that underlies the achievement gap.

The best data available for looking at the achievement gap over time is the long-term trend data from the National Assessment of Education Progress (NAEP). A national probability sample, NAEP data are detailed by age group, not grade, and since the test itself has remained stable since the early 1970s, it paints a picture of how things have changed over time.

What do the data reveal? Consider just reading performance among nine-year-olds from the year 1971 to 1999. [figure 1] Clearly, the achievement gap did not narrow over this period: into the 1980s, some progress was made, but from that point on, the gap stabilized. The situation is basically similar for mathematics and not so very different for science.³

In short, we have made progress—with the substantive improvements occurring early on—but not as much as we would like. It is instructive to note, however, that there are significant local variations: the gap reported in the state of Maine, for example, is much smaller (about one third of a standard deviation) than that in Wisconsin or Connecticut (both larger than one standard deviation). 

Mind you, a gap that measures one standard deviation represents a serious disparity in achievement. Moving a child who lands at the middle of the distribution up by one standard deviation would move him roughly from the 50th percentile to the 84th percentile—a change that would delight any educator.

How Important Is the Achievement Gap?
The typical contrast used to define the achievement gap is the black-white divide or increasingly the Hispanic-white. But the gap could be defined by socio-economic status, and it could be criterion-referenced or norm-referenced. Because people talk about the achievement gap in these various ways, we need to be precise about what we mean.

When does the achievement gap begin? The gap between whites and blacks is present before children experience any schooling. By the time children are three or four, it is already a standard deviation.

Does the gap increase while students are in school?  The surprising answer is no. Researchers have found that the rate of growth in achievement among blacks is equal to that among whites during the academic year. In the summertime, both groups show a decrease, but that decrease is larger for blacks than for whites. So while the achievement gap doesn’t increase while students are in school, it doesn’t decrease either.

Is the gap a function of test bias?  No matter how hard they have looked, researchers have been unable to find any evidence of test bias. A number of people have hypothesized that administering performance assessments rather than multiple choice achievement tests would show a smaller gap, but this is not the case. In fact, an achievement gap of one standard deviation on multiple choice tests increases to 1.2 standard deviations with performance assessment. My hypothesis about this finding is that black children on average are not receiving the schooling they need to acquire the kind of knowledge needed to succeed in performance assessments. That is, there is differential distribution of opportunity to learn between black students and whites.

How are definitions of comparison groups changing discussions of the gap? I would venture that in ten or 15 years, we won’t be talking about the black-white achievement gap. Since the current Census allows respondents to report multiple ethnicities, we will have a much harder time in defining ethnic groups in the future. The achievement gap will still be there, and we will still worry about it. But we will likely be worrying about it in terms of socio-economic status.

What Solutions Have Been Tried?
Since the 1960s, attempted solutions to this problem have generally fallen into four different categories: preschool reforms, teacher reforms, instructional reforms, and standards-based reforms.

Preschool Reforms. Almost all of the research on preschool programs shows early gains in achievement, and that the early gains are not sustained. Moreover, the academic advantages of preschool programs are less likely to be sustained for children of color than for white children. We don’t know why, but the finding has been replicated many times.

But these programs vary tremendously in quality. The Perry Pre-school evaluation famously found that particular program to be massively successful, with participating students half as likely to go into special ed, five times less likely to be incarcerated, four times more likely to earn $2,000 or more monthly. But the sad truth is that not all programs are good programs and, to make matters worse, white students are more likely to participate in preschools than their black peers and the schools they attend are more likely to be of high quality.

Teacher Reforms. Education research has finally caught up with common sense in its understanding of teacher quality. For a long time, everybody knew that a good teacher was better than a bad teacher, but no one could actually document that teachers made any difference. Now, researchers have documented teacher effectiveness in raising student achievement.

Say a student—call him Johnny—has a good teacher every year, in the first grade, second grade, right up through the 12th grade. Let’s say that the good teacher has the effect of improving Johnny’s performance one tenth of a standard deviation. So that at the end of the first year, Johnny is a tenth of a standard deviation better off than he otherwise would have been. Now let’s say that the shelf life of that effect is perfect (Johnny keeps that advantage when he goes to second grade). In second grade, he improves another tenth of a standard deviation. By the time Johnny graduates from high school, he’s 1.2 standard deviations better than he would have been—a difference bigger than the achievement gap.

The assumption that the advantage from one year to the next does not deteriorate over the summer months is not certain. But even so, the impact of teacher quality is powerful, and virtually everyone in the education community is convinced that the best reform would be an effective teacher in every classroom.

Would it close the achievement gap? Probably not. For an education reform to solve the achievement gap, it must produce bigger gains for black students than for white students. But most education interventions actually exacerbate the gap, and the more effective they are in raising mean achievement, the more they widen the gap. So if every teacher in every American classroom were effective, then all students—black and white—would have an effective teacher and student achievement across the board would rise. Closing the gap means instituting reforms that improve black students’ achievement at a higher rate than white students.

The research also confirms the effectiveness of other teacher reforms. In terms of teachers’ expectations of students, almost all the research shows that if teachers expect more of their students, their students will achieve more. Interventions designed to improve teachers’ expectations have shown modest effects.

Another intervention widely championed is the idea of black teachers teaching black students. Most results show that when black teachers teach black students, black students achieve more than when taught by white teachers. The policy implications are not straightforward. For example, schooling has many different goals—social and emotional ones as well as achievement. Even if the achievement gap would decrease, is it wise to have black students learning only from black teachers?

Instructional Reforms. With a million instructional interventions out there, let’s take one example—Success for All, a highly scripted intervention that can be implemented and replicated well. Rather than striving for excellence per se, Success for All focuses on raising the bottom level of achievement in classrooms. Many studies of this program find good effects—and greater effects, in fact, for black students than for white students. One could hypothesize that the intervention provides the opportunity to learn that black students tend to miss out on.

Another example of a popular instructional reform is reduced class size, but the results are mixed. The best study of class size—the Tennessee STAR study—demonstrated that reducing class size to 15 or below, a fairly major reduction, can have a good-sized effect on achievement in year one. In years two, three, and four, that first effect was maintained, but there was no additional advantage. I can’t think of a more expensive education intervention than this one, and its effect size is disappointing. Moreover, as California’s experience demonstrates, bringing reduced class size to scale can be a perilous task. When that state decided to reduce class size massively, they had to hire new teachers—many of them unqualified—and haul in trailers for classes. For one of the most expensive educational interventions out there, the impact of reducing class size leaves something to be desired.

Research on ability grouping and tracking delivers the counterintuitive news that it is enriched classes that tend to have positive effects on student achievement. Remedial classes, on the other hand, don’t have a negative impact but don’t provide much benefit either.

Following the release of A Nation At Risk in 1983, states increased their requirements for high school graduation and, in one way or another, have been continuing on that path ever since. Predictions were dire. First people foresaw a big decline in high school graduation rates, which didn’t materialize. Then, they were certain that enrollment in remedial courses would increase, and that didn’t materialize when students signed up for more college prep courses. Then, people predicted that teachers would dumb down those college prep courses—which also didn’t materialize: teachers continued to teach college prep courses as they always had.

And the effect on student achievement was huge. In one study, we looked at three contrasting ninth-grade mathematics curricula: Basic Math; Transition Math, which upgraded the curriculum but not to the level of college prep; and College Prep. Controlling for initial differences between students, the value added for student achievement was biggest for College Prep, followed by Transition Math. And then Basic Math was by far worst. Those findings represented a big success story for the idea that students benefit from being given an opportunity to learn at a higher level.

As for promotion and retention policies, the reviews are mixed. Because these policies are administered in so many different ways, evaluations of their effectiveness vary widely. The big hole in this research is that studies only compare kids who are retained versus kids who are promoted—they do not consider the changes to the system over time. Advocates of these policies argue that retaining students will be so painful to the system that it will be forced to improve. We don’t know whether that is happening but we do know that conducting the research to find out will be expensive.

Standards-based Reforms. The standards movement can claim some exciting accomplishments—most notably, putting student achievement on the map. Today, if you go to a school board meeting, if you talk to a superintendent or a principal or a teacher, you’ll hear people talking about improving achievement. Twenty years ago, those conversations had nowhere near the intensity they do now. Also on the plus side—at least for education researchers—is the focus on education research, on connecting research to practice.

But standards-based reform has been with us for ten or 15 years—first at the state level and now in the form of No Child Left Behind—and it does seem that by now, we would be seeing improvements that we’re just not seeing. But remember, too, that anything worth doing can be done poorly, and standards-based reform is no exception to that rule.

Addressing the Gap in Terms of Opportunity
Schools are not the major cause of the achievement gap. Long before kids go to school, the gap is alive and well, and, during the academic year when kids are actually in the classroom, it tends not to increase. Any increases that do occur take place largely outside the context of schooling.

Still, it is the schools we turn to for a solution. But we do well to remember that we are asking schools to solve a problem not of their own making. For schools to solve the achievement gap, we will need much more aggressive interventions—interventions that address the critical issue of opportunities to learn—particularly the opportunities we do (or don’t) provide to our most disadvantaged children.

The most promising reforms are alike in their attention to addressing the pervasive inequalities in opportunities to learn. Consider preschool. Done well, it shows some impressive effects, some lasting effects. But we need to make sure that the kids—all the kids—get this high-quality preschool. This is an opportunity to learn issue.

Consider teacher quality: the research shows that black students have less access to high-quality teachers than white students do and less access to good materials. This is an opportunity to learn issue.

Consider student course-taking patterns. The percentage of students taking college prep high school coursework is going way up for white students, for black students, and for Hispanic students. Over the last 20 years, the gap between black and white students in course-taking has dramatically reduced. This is an opportunity to learn issue—one where we have made real progress.

The achievement gap is unlikely to be totally eliminated by school reform. But that doesn’t get education off the hook. Some education reforms, especially those that provide greater opportunities to learn, do reduce the gap. High-quality preschool, effective teachers in every classroom, a challenging curriculum of enriched classes—all have been shown to have demonstrable effects on students’ academic performance and all have the potential to reduce the achievement gap.

Andy Porter is the dean of the Penn Graduate School of Education and the George and Diane Weiss Professor of Education.

Also from Andy Porter
To define and then measure alignment, researchers have been limited to looking at the alignment between the tests and the standards. Andy Porter and colleagues have developed several tools that provide a far more nuanced picture—one that examines the alignment between content standards, tests, textbooks, and even classroom instruction.

In this paper, Porter et al. discuss two of those tools—content maps and a quantitative index of the degree of alignment—and serve up a real-world example of their use. Using these tools, the researchers conducted a randomized trial to test the effects of a math/science professional development intervention on the instructional practices of middle school teachers.

Alignment as a Teacher Variable, by Andrew Porter, John Smithson, Rolf Blank, and Timothy Zeidner, appears in Applied Measurement in Education, 20(1).