Basketball, schools, and the rise of advanced measurement
Education is a lot like basketball.
Seriously. And we can learn something about what value-added measures (VAM), growth models, and other advanced statistical techniques through a look at the Atlanta Hawks, one of the best teams in the NBA.
First, some background. Prior to 2010, when fans wanted to use numbers to better understand the game, they had to rely on basic measures like points per game. The same was largely true for coaches and general managers, which made their jobs very difficult. Every basketball fan alive would agree that a player can do a million things that contribute to winning that don’t show up in traditional statistics. Some players do more of these invisible things than others, but no one had a way to measure most of them.
Advanced measurement has changed that. We now know more than ever before about which players contribute the invisible ingredients to winning. Instead of relying on how many points a player scores, we measure whether a team scores more points with that player in the lineup. To measure a player's defensive effectiveness, we measure his impact on the team’s defense as a whole.
These new, more accurate statistics have helped basketball fans better appreciate the value of players like Kyle Korver. Conventional stats show Korver, who scores about 13 points per game, to be a decent but unexceptional offensive player. Advanced stats help us realize his tremendous value to the Atlanta Hawks.
The NBA awards three points for shots 23 feet or further from the basket. To make even one such shot against an NBA defense is a small miracle. Korver makes 50 percent of his three point shots. As a consequence, NBA defenses have to account for him at all times, to the point that they routinely send a second man to help guard him. More defenders on Korver means fewer defenders close to the basket, which allows the rest of the Hawks far more opportunities for easy two-point shots. Advanced statistics shows Korver’s impact to be so pronounced that many seasoned NBA observers argued for him to make the All-Star team over established superstars like Dwyane Wade.
AEM’s mission is to help education better recognize its Kyle Korvers. For years observers have worried that measuring student learning through standardized test scores risks labeling students as failures, even if they make tremendous gains during the year, because they do not meet some arbitrary threshold score to pass.
Consider a hypothetical student's performance on the Virginia SOL exams. Possible SOL scores range from 0 to 600, with students required to score at least 400 to pass. On the Grade 4 Math test, our student scores a 342. A year later, on the Grade 5 Math test, she scores a 376.
Most of us would agree that our student has made tremendous strides. In one school year, she learned enough so that she scored 34 points higher on a more advanced test. These results probably mean that her Grade 5 teacher helped her catch up on much of the fourth-grade material and teach her enough fifth-grade material to move her that much closer to performing on grade level. Both the student and teacher should be celebrated.
Advanced statistical analysis allows us to do so. It allows us to predict the expected growth for each student so that we know when students have grown more quickly and to see which teachers spur better-than-average growth for multiple students. These techniques allow us to use standardized tests to create more fair standards for all students and teachers. We don’t have to scold children who put in yeoman’s efforts to improve but fall a few points short of an arbitrary passing score. We don’t have to measure teachers working in schools with more challenging populations against those who work in more affluent districts with children raised in environments that make success easier.
The negative implications of VAM and growth modeling dominate the debate far too often. The opportunities these methods create far outweigh the drawbacks. These techniques evaluate students and teachers on a more level playing field and create realistic, attainable goalsfor all.