Holding Tested-Based Accountability Accountable
I dug up something I wrote about a little while back and decided to post it in this space as it very nicely lays out my thinking on test-based accountability, something that I have been working through as I plan my strategy as the Education Program Director at Voqal.
Along with my experience in the field, two recent books deeply inform my thinking on the subject: : The Testing Charade by Daniel Koretz, and Beyond Test Scores, by Jack Schneider. Both of these books make strong arguments against our current test-based accountability system in favor of richer measures of school effectiveness. They are valuable additions to the research calling for better accountability measures and methods and bolster the growing groundswell to think differently about how we judge our schools.
Daniel Koretz is a noted scholar and one of the top experts on educational assessment and testing policy. The Testing Charade summarizes his over three decades of scholarship on assessment written for the lay audience (see him talk about it here). It is challenging to distill this important work down to a paragraph (but I’ll try anyway): Koretz argues that educational assessments are important and necessary, but we have tragically misused a blunt instrument in extraordinarily unrealistic ways to judge teachers, schools and the education system as a whole. The result is a system that has led to wide-scale manipulation, cheating, and the general degradation of teaching and learning in the name of so-called accountability. He offers a useful and possible plan for doing better, but this will necessarily call for extensive trade-offs regarding how we evaluate and hold schools accountable.
Jack Schneider takes this a step further by offer a powerful framework for fairly and comprehensively measuring school quality based on a multi-year project undertaken in the highly diverse urban school district of Somerville, Massachusetts. This framework calls for measuring a wide-range of Essential Inputs (Teachers and Teaching Environment, School Culture, and Resources) in addition to Key Outcomes (Academic Learning, and Character and Well-Being) to evaluate schools. By collecting relevant data, analyzing them appropriately, and presenting them to teachers, administrators, policymakers, students and parents in accessible and useful ways, a richer and more accurate picture of school effectiveness develops. This provides all education stakeholders with the tools and information to better judge school quality and build a more accurate and actionable system of school accountability.
What both of these books stress very clearly is that educating humans is extremely nuanced and complicated, and schools are complex ecosystems. Neither can be accurately or confidently evaluated through a single instrument, no matter how refined and validated that instrument is. This is important. Standardized assessments are critically important to provide a snapshot of how some academic learning is progressing in a school, but no standardized test can capture more than a relatively small fraction of what is actually going on in a school. And they certainly cannot by themselves tell us how good (or bad) a school or teacher is at helping kids learn.
Education is about much more than teaching children how to read, write, and do math; it is also about helping children grow socially and emotionally so they can become engaged and empowered citizens and well-rounded adults who can confidentially navigate the increasing complexity of our modern world. In fact, the overwhelming evidence from brain science, medicine, economics, psychology and education research confirms that learning occurs through an interconnected focus on social, emotional, and cognitive development. Yet, our current test-based accountability system really only examines cognitive development, and only snippets of development at that. We are missing major pieces of the learning puzzle, but we currently use this incomplete information to make extremely high-stakes decisions concerning educators, schools and students (note: the current version of the federal education law, the Every Student Succeeds Act (ESSA), represents a slight shift away from this, but there is a long way to go).
More importantly, as Koretz so robustly documents, the use of academic tests as the primary accountability method is not only seriously flawed, it is educational malpractice. The core of this argument comes from Campbell’s Law, which states: The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. The current misuse of test-based accountability and resulting unintended negative consequences unfortunately follows this law to the letter.
I experienced a radically disparate and inequitable public education system in NYC at a time when accountability meant little. I’ve been working in the education sector for over fifteen years and I want to see it improve, especially for those that have been historically underserved. And I believe in holding schools and educators accountable for whole student learning. But this doesn’t mean having a blind or unrelenting faith in an accountability system or any reform silver bullet — and there are a lot of these — that is not helping drive the outcomes we want (or a least should want). I want better for our kids, especially when what we have isn’t effective.
I am cautiously optimistic about the promise and potential for thinking bigger about measuring school quality. By shifting away from faulty test-based accountability in favor of wide-ranging measures and robust use of new data, we can better understand what makes a quality school, determine what resources schools need to be successful, and evaluate teaching, learning, and whole child development through a richer set of outcomes. It’s not easy or cheap, and will call for some significant disrupting of our current accountability practice. But I am convinced that it is possible. And worth it.
