JKCF in the News: Education Officials Flunk Statistics 101
Education Officials Flunk Statistics 101
‘Big data’ analysis provides insights into everything from school attendance to the progress of talented students.
By HAROLD O. LEVY
Sept. 14, 2014 5:11 p.m. ET
As students return to school this fall, some basic math may come as a surprise: The data that officials employ to judge students and schools is misunderstood, ignored and or misused, particularly when measuring the performance of low-income students.
There is, however, reason for optimism. New “big data” methods for better analyzing more information are improving education research, forcing administrators, teachers, students and parents to question long-held assumptions.
Consider truancy. Nearly all school districts proudly report around 90% average daily attendance. That statistic is misleading. It might seem like only one-in-ten students is out on any given day, but that is not really the case. Some students always show up; others miss the bell often. A school district could report 90% average attendance and still have 40% of its students chronically absent, as a 2012 report by Johns Hopkins researchers Robert Balfanz and Vaughan Brynes showed.
A closer look at the numbers reveals that truant children miss about a month of class every year. As they advance from one grade to the next, their learning gaps eventually become so pronounced that they can’t keep up. These children run a much greater risk of dropping out, eliminating opportunities for the rest of their lives. The situation is worst in low-income and minority neighborhoods. In California, African-American students were truant last year at a rate more than 2.5 times their white counterparts, according to a report released on Friday by the state’s Attorney General Kamala Harris.
Call it the “broken window” theory of student performance. Just as crime went down when cops realized that addressing minor infractions such as broken windows prevented bigger crimes like muggings, so too education officials could help children stay in school as teenagers if they made attendance a bigger priority earlier.
Another area where education officials flunk Stats 101 is how we treat high-performing, low-income students. A study by the organization I run, the Jack Kent Cooke Foundation, used Education Department data to track talented students. Only 59% of smart children—those who scored in the top 25% on standardized tests—from low-income households graduate from college. But 77% of similarly bright children from wealthy families finish an undergraduate degree. By one measure, high-achieving students from low-income backgrounds graduate from college at about the same rate as low-scoring students from affluent families.
Yet school leaders and politicians have done nothing to help these high-achievers prepare to continue their success in college, although their performance is all the more impressive considering the challenges they face. The federal government’s only program supporting gifted students—the Jacob Javits Gifted and Talented Students Education Act—was appropriated a grand total of $5 million in 2014 and nothing in the previous three years.
Arguably the most prominent example is the movement to hold students back if they fail to meet state standards. When around 2000 the “no social promotion” policy, as advocates such as Rudy Giuliani called it, gathered momentum, it sounded reasonable. Today, six states have enacted or are about to adopt this “tough love” policy.
Here’s the problem: No independent academic study suggests it works. Repeated studies have shown that being held back destroys a student’s confidence, and once left back that student is far more likely to drop out than a similar student who was promoted. Those who have been held back are twice as likely as comparable peers to repeat a grade for a second time, according to a 1996 study by the Texas Education Agency.
New studies using innovative analytical techniques have told us even more. Research on summer school, for instance, has in the past shown that the extra weeks did little to improve student performance. But in 2007 Cornell University economist Jordan Matsudaira used data to create a “synthetic control group” composed of thousands of statistically identical students, those who came from the same socioeconomic background with the same test scores.
Mr. Matsudaira found that students who attended summer school performed better and were less likely to drop out. It had a similar effect as cutting class size down by a third. In fact, the researchers concluded that summer programs are among the most cost-effective intervention strategies for the lowest performing students.
But this finding has not prompted a resurgence of summer programs, just as the compelling data on the lower college-graduation rates of high-achieving students from poor backgrounds has not led to helpful policy changes. It’s rare to find a school administrator who is comfortable parsing statistics or stays ahead of the academic literature. So these insights go undeveloped, and students who could be saved are not.
Perhaps teachers, administrators and bureaucrats will pay attention as new data in education proliferates, but today’s struggling students can’t wait that long. The lag between what we already know and how we teach is robbing children of their future, and undermining the idea that America is a meritocracy.
Mr. Levy, executive director of Jack Kent Cooke Foundation, which provides scholarships to exceptionally high-achieving students from low-income families, was New York City schools chancellor in 2000-02.