Nimble Assessments for Developing Countries
In a West African schoolroom, a student carefully copies words from the blackboard. The problem is, she can’t read them.
In the world’s poorest countries, scenes like this play out again and again. In Malawi, for example, as few as one percent of sixth-graders can read a complex text with understanding. In Zanzibar, the number is five percent.
Over the past few decades, explains Penn GSE Professor Dan Wagner, the international community has focused on improving literacy and the quality of education for children in the developing world. A succession of high-profile events – the World Conference on Education for All in 1990, a follow-up meeting in 2000, the adoption of the UN Millennium Goals in 2000 – all signaled a growing commitment to the issue.
Like policymakers in the U.S., international donor agencies and NGO’s want to see hard data about student achievement in the countries where they work. To get that information, they urge the adoption of large-scale international assessments designed to provide hard comparative data about educational attainment across countries – tests like TIMSS (Trends in International Mathematics and Science Study) and PISA (Program for International Student Assessment).
But are these international assessments, geared as they are to developed economies and educational systems, the best choice for low-income countries?
“What am I going to do with these large-scale assessments after I know that we’re still at the bottom of the heap?” asks Wagner. An expert in international education development, Wagner finds merit in a different model – one he calls the “Smaller, Quicker, Cheaper” option.
More informal and smaller in scale than the international behemoths, this new breed of assessment (SQC, for short) is comparatively nimble. Since they have a smaller sample and a faster turnaround, these assessments enable educators to craft interventions almost in real time. Some of the large-scale international assessments can take as long as eight years to post results. With SQCs, a ministry of education can get feedback to teachers and principals within a year, or even less.
Small in scale, these tests are generally sample-based and localized (focused on a specific country or even region). They typically sample kids at a younger age (grades 1 through 4) so that educators can respond in the critical early years of schooling.
In addition, assessments like PISA favor international languages, but SQCs can be constructed in the local language and thus provide opportunities for early intervention for some of the most disadvantaged students.
What’s more, the big international assessments carry a big price tag, requiring top talent to devote time and resources to comprehensive analysis. But a country with only one resident expert in the field will have to devote all of his expertise just to that test for five years or more.
Take Rwanda, says Wagner. “They really care, but they can’t participate in the bigger study. They don’t have the resources to do it. But they are interested in knowing what is happening regionally… so they’re doing focused population samples across the country but not trying to do a national survey at this point.”
As Wagner points out, different assessments have different policy and practical implications. “One size does not fit all,” he says, and developing countries need to take a careful look at their assessment needs and choose accordingly.
What’s at stake is vital to their future, says Wagner. “Learning levels matter for economic development,” he explains. “In the literacy field, we find correlations between adult literacy rates and just about every good social outcome – GNP per capita, lowered infant mortality, improved life expectancy, lower HIV/AIDS.”
Dan Wagner, who holds the UNESCO Chair in Learning and Literacy at Penn GSE, spoke on “Quality of Education, Comparability, and Assessment Choice in Developing Countries” this spring.
You can watch his lecture here.
For more on the topic, read his article in Compare: A Journal of Comparative and International Education.