Research Notes - Fall 2010

Penn GSE faculty and researchers explore the issues at the forefront of American education, engaging in high-impact research, innovation, and training in public education, as well as in literacy, psychology, social policy, and higher and adult education.

For summaries of their recent studies and findings, click on the titles below.

Early Childhood Risks and Educational Outcomes

Two recent studies conducted by John Fantuzzo and colleagues looked at the impact of early childhood risks on the educational outcomes for diverse, low-income children.

Two recent studies conducted by John Fantuzzo and colleagues looked at the impact of early childhood risks on the educational outcomes for diverse, low-income children.

The first study examined multiple maternal risks on the competencies of diverse, low-income preschoolers and found that a mother’s education accounted for the most variance in children’s cognitive outcomes while chronic maternal depression accounted for the most variance in their pro-social and problem behaviors. Further findings revealed that different combinations of maternal risks were associated with varying outcomes.

The second focused on an entire cohort of second-grade students in a large urban district to understand the timing and influence of the first experiences of maltreatment and homelessness on academic achievement and attendance. Significant among the patterns that emerged were the power of timing (the earlier the exposure, the worse the outcome) and the relative impact of these risks (maltreatment has a more pervasive impact than homelessness).

"A Multivariate Investigation of Maternal Risks and Their Relationship to Low-Income, Preschool Children’s Competencies," by Marlo A. Perry and John W. Fantuzzo, appears in Applied Developmental Science, 14(1).

"Timing and Influence of Early Experiences of Child Maltreatment and Homelessness on Children’s Educational Well-Being," by Staci Perlman and John Fantuzzo, appears in Children and Youth Services Review, 32(6).


Also from John Fantuzzo

The National Research Council has called for researchers to investigate the use of assessment measures in high-stakes evaluations of programs for young, low-income, minority populations. One of those assessments, the Child Behavior Checklist (CBCL), was originally developed to detect psychiatric syndromes as observed in a largely middle-class, largely Caucasian sample. To determine whether an assessment with that pedigree could be effective when applied to a non-clinical setting and a low-income, minority population, a Penn GSE team examined data from an evaluation of an early intervention that had used the CBCL to assess program impacts on the behavioral adjustment of low-income children. The Penn study found no support for the relevance of CBCL to this particular population; indeed, less than one third of the clinical behaviors included in the CBCL were prevalent in this community-based sample. "Measurement and Population Miss-Fits: A Case Study on the Importance of Using Appropriate Measures to Evaluate Early Childhood Interventions," by Whitney A. LeBoeuf, John Fantuzzo, and Michael L. Lopez , appears in Applied Developmental Science, 14(1).

Faculty Unions and Academic Governance

Of comparatively recent vintage, faculty unions feature prominently in American higher education, particularly in community colleges. In a recent qualitative study of the governance system at a large, urban community college, Matthew Hartley found that, although relations can indeed be contentious, positions are not static, but exist along a continuum.

Of comparatively recent vintage, faculty unions feature prominently in American higher education, particularly in community colleges. When they entered the scene in the 1970s, many predicted the end of collegial shared governance. Although experience seems to suggest that these predictions were a tad overblown, little institution-level research has looked at how unionization has in fact affected academic governance.

In a recent qualitative study of the governance system at a large, urban community college, Matthew Hartley found that, although relations can indeed be contentious, positions are not static, but exist along a continuum. In some negotiations, posturing and acts of brinksmanship can put a serious strain on relations between the administration and the union. Other situations play out more like a chess match, with both sides maneuvering for advantage. Still other decisions are made in a spirit of mutual respect and collegiality.

Hartley identified a number of factors that mitigated conflict and promoted collegial decision-making, among them, drawing on personal ties between individuals, making collective sense of shared data before discussing possible resolutions, and establishing an institutional culture that values finding solutions rather than scoring political points.

"Reconcilable Differences: Conflict and Collegiality in a Unionized Community College Environment" appears in Community College Journal of Research and Practice, 34(4).

Also from Matthew Hartley

In examining the civic engagement movement in American higher education over the past three decades, Hartley focuses on the critically important (and frequently overlooked) role that leaders have played in advancing these efforts. The article describes how leaders attempted to build networks and points to the challenges of negotiating ideological tensions that inevitably emerged as the movement evolved.

"Leading Grassroots Change in the Academy: Strategic and Ideological Adaptation in the Civic Engagement Movement" appears in Journal of Change Management, 9(3).

Hampton Institute and Post-Brown Fundraising

In the wake of the Brown v. Board decision, many observers wondered about the long-term viability of Black colleges. Their enrollments threatened by the push toward integration, these institutions suffered as well from declining support from corporate and foundation funders.

In the wake of the Brown v. Board decision, many observers wondered about the long-term viability of Black colleges. Their enrollments threatened by the push toward integration, these institutions suffered as well from declining support from corporate and foundation funders. Such was the terrain when the Hampton Institute launched its largest fundraising campaign ever—and, in a related move, withdrew from the United Negro College Fund (UNCF).

The campaign, which focused on racial uplift and future integration, was launched in 1964; the decision to leave the UNCF came four years later. Among the consequences for Hampton were the need to maintain strong corporate and foundation ties, to increase faculty, staff, and student annual fund participation; and to build alumni giving. For several years, the institution struggled to replace the UNCF funds but then prospered, today boasting an endowment of $186 million, a balanced operating budget, and an alumni giving rate of 15%.

"Fundraising for Black Colleges during the 1960s and 1970s: The Case of Hampton Institute," by Marybeth Gasman and Noah Drezner, appears in Nonprofit and Voluntary Sector Quarterly, 39(2).

How to Plan Ahead

One of the consistent findings of education research is the inconsistent quality of program implementation. To determine whether we can predict which parts of a program will stick and which will slip, Penn GSE researchers conducted a three-year study of five school-improvement programs in 15 high schools.

When it comes to school reform, there’s many a slip between program design and improved student achievement. One of the consistent findings of education research is the inconsistent quality of program implementation.

To determine whether we can predict which parts of a program will stick and which will slip, Penn GSE researchers conducted a three-year study of five school-improvement programs in 15 high schools. Finding substantial variation in how programs were being implemented, the researchers theorized that implementation is a process of iterative refraction—that is, reforms are adjusted repeatedly as they make their way through schools.

The theory further suggests a certain predictability in the implementation process. The researchers identified three key "junctures"—that is, points where reform is especially vulnerable to modification—and suggest that designers and change agents pay particular attention to them as they develop and roll out new programs.

First is the design stage: To guard against refraction, designers can employ specific strategies: emphasize the "non-negotiables" in the reform; calibrate the level of school engagement required to the level of complexity; consult directly with schools and school personnel; and support for change.

Next comes the school level, where leaders—both formal and informal—can make or break a reform. Without the legitimizing support of officialdom, reforms often wither on the vine, and informal leaders—usually teachers without formal titles but robust networks—wield a great deal of influence as well.

Finally, the central office plays a key role. Specific issues at the district level were the nature of its relationship with the high school, its human and fiscal capacity, and its ability to align its own operations and the reform’s priorities.

"Planning Ahead: Make Program Implementation More Predictable," by Elliot Weinbaum and Jonathan Supovitz, appears in Phi Delta Kappan, 91(7).

Interim Assessment and Instructional Change

For its fans, interim assessment holds out the promise of improved student learning through improved instruction. Skeptics wonder whether the optimism is unwarranted, resting largely on evidence about short-cycle formative assessments.

A recent study, conducted by Penn GSE researchers, explored whether teachers are indeed using data gleaned from interim assessments to make instructional change.

For its fans, interim assessment holds out the promise of improved student learning through improved instruction. Skeptics wonder whether the optimism is unwarranted, resting largely on evidence about short-cycle formative assessments.

A recent study, conducted by Penn GSE researchers, explored whether teachers are indeed using data gleaned from interim assessments to make instructional change. Focusing on 45 elementary school teachers in two districts (Philadelphia and one of its suburbs), the study asked how these teachers gathered, analyzed, and acted on data about student learning. The researchers also considered factors like district policies, organizational norms, and individual educator capacity.

Study findings suggest that, while often useful for teachers, interim assessments aren’t sufficient to improve instructional practice. Although they can—and do—help teachers decide what content to re-teach and to whom, they don’t help develop a deep understanding of students’ learning, a prerequisite to real instructional improvement.

Most assessments provided little actionable information on student’s misunderstandings—a deficiency compounded by teacher uncertainty about just how to interpret data. As the researchers observe, "many teachers focused on procedural rather than conceptual sources of student errors on test items, diagnoses that appeared to inform their instructional planning during re-teaching."

Based on these findings, the researchers recommend designing assessments specifically for instructional purposes; providing effective support for their use; and building teacher capacity on interpreting and applying data.

"Can Interim Assessments Be Used for Instructional Change?" by Margaret Goertz, Leslie Nabors Oláh, and Matthew Riggan, is a CPRE Policy Brief based on a CPRE Research Report From Testing to Teaching: The Use of Interim Assessments in Classroom Instruction.

More on Interim Assessments

To learn more about how teachers use data from interim assessments to modify instruction, Leslie Nabors Oláh, Matthew Riggan, and Nancy Lawrence interviewed third- and fifth-grade teachers in five average- and above-average-performing Philadelphia schools. How were teachers benchmarking assessment results? What were their instructional responses? The teachers were found to use interim assessment results for insight into students’ learning, but not into their grasp of concepts. Likewise, mirroring this tendency to downplay conceptual understanding, their instructional responses focused on procedural issues.

"Learning to Learn from Benchmark Assessment Data: How Teachers Analyze Results" appears in Benchmarks for Success? Interim Assessments as a Strategy for Educational Improvement, a special issue of the Peabody Journal of Education, edited by Leslie Nabors Oláh, Katrina Bulkley, and Suzanne Blanc.

It Takes Two

In pair therapy, two youth work together toward reciprocal transformation. An analysis of two-year pair therapy of two preadolescent boys gave GSE's Mike Nakkula the chance to observe reciprocal transformation in action.

In pair therapy, two youth work together toward reciprocal transformation. Facilitated by adults, the sessions help youth develop strategies for sharing experiences and resolving conflicts—skills they’ll need to sustain deep relationships.

Working from the assumption that negotiating differences provides the catalyst for change, the technique favors matching youth with substantially different personal styles. The two basic styles, Mike Nakkula explains, are "self-transforming" and "other-transforming"—defined respectively as focusing on changing oneself to avoid conflict and attempting to change others to meet one’s own needs.

An analysis of two-year pair therapy of two preadolescent boys, Alex and Jamie, gave Nakkula the chance to observe reciprocal transformation in action. From its "honeymoon phase," the therapy proceeded through a cooling-down period characterized by co-existence rather than full engagement; reached a crisis when the self-transforming Alex directly challenged Jamie’s aggressive behavior; and resolved in a rebalancing of power and a deepening sensitivity to each other’s concerns.

Commenting on the "relational risk-taking" involved in the model, Nakkula observes that adults too often attempt to forestall risk-taking among young people—often to the detriment of growth promotion.

"Transforming Self-Control Through Peer Relationships" appears in Reclaiming Youth at Risk, 17(4).

Low-Income Students, Financial Aid, and College Choice

In the United States, the stratification of college choice by family income persists—even in light of the investment elite colleges and universities make in student aid. Low-income students are more likely to enroll in a private for-profit or two-year college than in one of the most selective institutions.

A recent analysis from Penn GSE explored how institutional financial aid influences the college choices of this population.

In the United States, the stratification of college choice by family income persists—even in light of the investment elite colleges and universities make in student aid. Low-income students are far more likely to enroll in one of the nation’s private for-profit or two-year colleges than in one of its most selective institutions, private or public.

A recent analysis from Penn GSE explored how institutional financial aid influences the college choices of this population. With a particular emphasis on schools that have eliminated the use of loans for low-income students, the study homed in on communicative strategies with a review of the websites of each no-loan school.

Among the colleges and universities with generous aid policies, the authors struggled to find information about aid packages. "One of the most striking findings from our review of institutional websites," they write, "was how difficult it was for our team of five individuals with a high degree of knowledge of higher education to tell if an institution had a no loan policy."

Assuming that these institutions find the benefits of enrolling more low-income students to be worth the cost of expensive aid packages, the authors suggest several remedies: more effective and more targeted communication strategies (both for students and their school counselors); additional staff to provide support through the financial aid process; partnerships with pre-college programs; and post-enrollment support for students from lower socioeconomic backgrounds.

In addition, institutions might do well to consider the limits of no-loan policies and, rather, consider putting "a thumb on the scale" for this population—much as they do for underrepresented minorities, children of alumni, and athletes.

"Showing Them the Money: The Role of Institutional Financial Aid Policies and Communication Strategies in Attracting Low-Income Students," by Laura Perna, Valerie Lundy-Wagner, April Yee, Leykia Brill, and Teran Tadal, appears in Recognizing Social Class and Serving Low-Income Students in Higher Education: Institutional Policies, Practices, and Culture, edited by Adrianna Kezar.

Also from Laura Perna

Using data from the Integrated Postsecondary Education Data Systems, Laura Perna et al. examined the status of equity for Hispanic students in public higher education institutions in Florida and Texas. Their analysis revealed substantial inequities across the board—in student enrollment, full-time faculty, and administrative and managerial staff. "The Status of Equity for Hispanics in Public Higher Education in Florida and Texas," by Laura Perna, Chunyan Li, Erin Walsh, and Stephanie Raible, appears in Journal of Hispanic Higher Education, 9(2).

Measuring School-Level Mental Health Capacity

In the U.S. today, 70 percent of psychosocial services for children take place in the schools, but assessments of these programs have focused narrowly on individual outcomes rather than on institutional effectiveness. To fill that gap, Caroline Watts and colleagues have developed the School Mental Health Capacity Instrument

In the U.S. today, 70 percent of psychosocial services for children take place in the schools, but assessments of these programs have focused narrowly on individual outcomes rather than on institutional effectiveness. To fill that gap, Caroline Watts and colleagues have developed the School Mental Health Capacity Instrument (SMHCI), which assesses policies, systems, and activities as they relate to the levels of the public health pyramid: intervention, early recognition, and prevention. Administered to all pertinent school personnel, the SMHCI includes 27 items designed to assess the structures school has in place to address student mental health needs.

Piloted in 13 urban schools (nine elementary and four high schools), the instrument also underwent validity testing; the six expert raters recruited for that purpose provided evaluations that evaluations matched up with the SMHCI scores.

"The SMHCI appears to provide a way to quantify this notion of mental health capacity," the authors conclude. "Further results help locate where a school is along a continuum from reactive to proactive...."

"The School Mental Health Capacity Instrument: Development of an Assessment and Consultation Tool," by Luba Falk Feigenberg, Caroline Watts, and John Buckner, appears in School Mental Health.

Racial Differences and Educational Expectations

Key to predicting students’ educational attainment is understanding their educational expectations. In a recent quantitative study, Shaun Harper and colleagues examined the process by which high school students from various racial backgrounds cultivate and reformulate those expectations.

Key to predicting students’ educational attainment is understanding their educational expectations. Socio-economic status, school quality and curriculum, grades and test scores, interpersonal influences, and students’ perceptions of their own abilities—all are known to affect educational expectations. A growing body of evidence suggests that race, too, plays an important role in students’ expectations for themselves—and in their parents’ and teachers’ expectations as well.

In a recent quantitative study that drew on data from the National Education Longitudinal Survey, Shaun Harper and colleagues examined the process by which high school students from various racial backgrounds cultivate and reformulate their educational expectations. Specifically, they looked at how various factors affect students’ expectations directly and indirectly and how these effects vary across groups.

Study results confirm that educational expectations do indeed vary across racial groups. They also found that different factors had different impacts on different groups. While the reasons for these differences are not always clear—why should fathers’ expectations have more impact on some groups than mothers’ expectations?—the authors suggest that schools can use their findings to adapt their practices to serve more students better.

"Racial Differences in the Formation of Postsecondary Educational Expectations: A Structural Model," by Samuel Museus, Shaun Harper, and Andrew Nichols, appears in Teachers College Record 112(3).

The Principal Impact

Amidst calls for increasing levels of educational accountability, a team of Penn GSE researchers set out to explore the connections between principal leadership and peer influence on both teachers’ instructional practice and levels of student learning.

Amidst calls for increasing levels of educational accountability, a team of Penn GSE researchers set out to explore the connections between principal leadership and peer influence on both teachers’ instructional practice and levels of student learning.

Principals can influence their schools in many ways: by focusing institutional mission and goals, by encouraging an environment of collaboration and trust, and by actively supporting teachers’ instructional improvement. Similarly, teachers are often influenced by their peers—through formal and informal instructional advice networks, observations of others' teaching, and opportunities to collaborate.

Using data from two teacher surveys—that measured changes in their instructional practices and their perceptions of principal leadership—and student achievement outcomes, Jonathan Supovitz, Philip Sirinides, and Henry May found evidence that, through their impact on teachers’ instructional practices, principals have significant effects both on teachers’ instructional practices and on teacher peer influence. Further, the peer influence of other educators was also found to be an important predictor of a teacher’s change in instruction for both English language arts (ELA) and mathematics.

Principal leadership and peer influence on teaching reveal themselves differently depending on the subject matter. While the impact of both was similar in ELA, the impact of peer influence on mathematics was nearly twice that of principal leadership, leading the authors to speculate that in subjects where principals are less comfortable, teachers are encouraged to fill the void of support and assistance.

In finding that peer influence also had a strong impact on instructional practice, this study provides some of the first empirical evidence for claims made by advocates of teacher leadership. It also points to the critical importance of the principal. Past literature has suggested that a principal’s main impact is on teachers who interact with students directly, but the authors’ findings indicate that principal influence is much broader.

As for the impact on student learning, the authors found that, at least in ELA, principal leadership is significantly related to student performance. Although they spend little time in the classroom, principals can have a vast impact on student learning through fostering a school-wide climate of instructional collaboration.

"How Principals and Peers Influence Teaching and Learning" appears in Educational Administration Quarterly, 46(1).