Understanding KIDS

By Nancy Brokaw

What if, at the push of a button, you could find out how the very first days of a child's life affect how he or she makes that critical early adjustment to school? What if you could see the impact of low birth weight or teen parentage on a kid's early literacy skills? And what if you could control for those risk factors to reveal the difference that programs like Head Start can make?

In Philadelphia, those are precisely the kinds of questions researchers are setting out to resolve. Of course, their answers don't come at the push of a button or the click of a mouse. Rather, they come from an archive of linked data, called the Kids Integrated Database (KIDS), that gives researchers access to just about everything the city of Philadelphia knows about its young children.

And the city knows a lot. Municipal agencies amass a wealth of administrative data-population-based information about public programs and the people they serve-that can, when integrated, tell researchers and policymakers volumes about what works and what doesn't.

The idea behind KIDS is breathtakingly simple: to link the records of individual children in the various databases maintained by separate city agencies. Funded by the William Penn Foundation and created by three Penn professors-Cartographic Modeling Lab Director Dennis Culhane, Penn GSE Professor John Fantuzzo, and Center for Mental Health Policy Director Trevor Hadley-KIDS links at least six databases maintained by the Department of Public Health, the Department of Human Services, the Office of Emergency Shelter and Services, and the School District.

Is Big Brother Watching?
"What this project has done is to create a single mechanism," explains Culhane. "Previously, researchers had to go to each of these agencies individually. The main difference now is that we are taking a systematic approach instead of pursuing these data requests on an ad hoc basis-so that the process doesn't have to be reinvented every time."

On the surface, that's a simple idea, but its execution was no small achievement. For starters, Culhane and his colleagues at the Cartographic Modeling Lab had to design the mechanics of the system, figuring out a way of linking data created at different times, by different people, for different purposes.

For example, all four School District databases (attendance, achievement, standardized testing, and special education) assign children with a single identifier that follows them through their entire school career, but that kind of transparency is the exception, not the rule, at City Hall. To create a unified system, programmers wrote an algorithm that assigned to each individual a unique identifier that was applied systemwide to that particular young person. Once they were done, they'd built an infrastructure, a system of linkages that, in effect although not in fact, merges information from more than a half dozen sources.

With a system as powerful as KIDS, concerns about Big Brother intruding into the private lives of citizens were inevitable. After all, much of the material is deeply personal-health records and details about family life and accounts of social service interventions.

As Culhane explains, such concerns arise largely from misconceptions about how sensitive data are handled. Each agency has its own regulations regarding identified information, and anyone making use of that has to observe those regulations to the letter. It helped that Penn researchers-including everyone on the KIDS team-have been working with city databases for years and thus have an insider's understanding of each agency's protocol for ensuring confidentiality. Nonetheless, each agency required that both the KIDS project team and any researcher who makes use of the system sign its particular confidentiality agreement.

Better Than Government Work
But, as anyone familiar with the complexities of municipal government will understand, Culhane had landed the easy job.

Negotiating the project with the city meant that the KIDS team had to make some compelling arguments about its potential value to City Hall. For one, the system would provide immediate payoffs by enabling the city to track its services and interventions closely-showing, for instance, when kids were being overserved and when they were falling through the cracks. Equally attractive were the down-the-road benefits the city would enjoy in the competition for federal and state funds, where understanding the effectiveness of interventions is critical: with KIDS, Philadelphia would be the only municipal jurisdiction in the country set up to provide that kind of information.

Nonetheless, the KIDS team had to negotiate a lot of red tape. To forge a workable agreement, Fantuzzo and his colleagues had to devise a way of satisfying federal regulations that prohibit agencies from integrating data unless that information serves an important administrative purpose.

The solution they found reflects their commitment to building a dynamic and truly collaborative process that will survive over time-and that will put the needs of the agencies at the heart of the research agenda. They located decision-making with a Policy Review Committee, whose members are recruited from each participating agency. Researchers looking for access to KIDS have to make a compelling case to the committee that their particular project addresses a concern of real importance to the city.

Fantuzzo explains, "The committee provides a safeguard that research will not be conducted for esoteric purposes, but rather that it will be important, relevant, and with clear policy implications."

Moreover, he continues, "The Policy Review Committee sets up a dynamic process in which researchers and high-level policymakers have conversations with one another. It creates an important, ongoing transaction between researchers and the frontline people responsible for the welfare of the children in the city."

In fact, it's that promise of collaboration-of the university and the city talking to one another about how to improve the lives of Philadelphia's children-that gets Fantuzzo most excited about KIDS. "We have all this information about young children in these silos," he explains, "and the agencies that collect it all have mandates to provide data to Washington. But it's all government work.

"What we did was to ask, ‘Who is generating the research agenda? Why can't the data be used to talk about child well-being?' We want the people who collect the information to be involved in creating the agenda."

A Comprehensive Case Study
As a research tool, what is most impressive about KIDS itself is its extraordinary descriptive power. The infrastructure that Culhane's lab built enables Fantuzzo's researchers to paint a richly detailed picture of a particular cohort of children through their interactions with official Philadelphia.

Take the Early Childhood Experiences and School Success study. The study had its beginnings in 2000, when a team of Penn GSE researchers began to take a close look at the preschool experiences of the children entering kindergarten in the School District of Philadelphia. When Fantuzzo released their first findings in September 2002, he described them as "an evidence base for hope." And, indeed, what his research team had found was an impressive vindication of the value of formal early learning experiences: kids who had attended Head Start and other preschool programs maintained higher skills throughout the year, achieving habits and motor skills that are more advanced than those of their peers.

Fast forward a couple of years, and that same project is still chugging away: each year, as a new crop of youngsters makes its way into kindergarten, their teachers fill out a questionnaire that details the students' preschool experiences, giving Penn researchers an abundance of data that will help address the question of whether the differences found in Kindergarten hold in grade 3.

But now, with KIDS at its disposal, the project team can look back into a child's very first years. That retrospective examination will enable researchers to paint a more complete picture of children's lives before they even reach the District, detailing risk factors like teen parentage, low birth weight, placement out of home, etc.

Says Heather Cohen, a Penn GSE doctoral student collaborating with Fantuzzo on this project, "We already know that kids benefit from formal preschool, but with KIDS, we can look at specific questions about the impact of formal programs." Do certain populations of children-kids living in single-parent homes, for instance, or special ed kids-take particular benefit from programs like Head Start?

In a similar study on the impact of foster care on school adjustment, Penn GSE doctoral student Staci Peckham and Fantuzzo are hoping to pinpoint which services are associated with resilient outcomes-and to shed some light on the question of what works for children whose lives have become entangled in the foster care system. Yet another set of researchers is looking at special needs children being served by both the behavioral health and the special education systems in an effort to determine which combinations of services result in the best educational outcomes for this vulnerable population.

Once completed, this trio of evaluations-all on high-priority issues-should serve as a comprehensive case study to demonstrate how KIDS can help answer previously unanswerable questions. While all three studies are local in focus, the KIDS team hopes that their impact will extend far beyond the banks of the Schuylkill River.

Says Fantuzzo, "KIDS is an infrastructure for creating good multidimensional intelligence about kids and about how city governments are succeeding in helping them-and it does so without creating a new billion-dollar program. I'd like to see the day when people are asking us, ‘How did you navigate through all the regulations? How did you build trust? How did you do this?' I'd love to have people coming to us to find out how we did it so they can replicate it in their community."

This article is reprinted from @Penn GSE: A Review of Research, Fall 2004.