I noticed a couple of months ago on Twitter a few retweets about an article The Standard, which is the journal of the National Association of State Boards of Education (NASBE), discussing dashboards. Robert Rothman tweeted a link to the an article titled "Accountability for What Matters". I was definitely intrigued, so I read the article. I sent a quick tweet back at Mr. Rothman saying I thought the article was interesting, but seemed to conflate the meaning of accountability systems, performance management, and dashboards. He didn't respond, so I thought I thought I would clarify what I mean in a blog entry.
First, let me unpack what Rothman means by "dashboard." Rothman writes:
With their newfound authority under the Every Student Succeeds Act (ESSA), states will be using a broader range of indicators of school performance and displaying them in ways that give school communities, parents, and district and state officials a clearer picture of how a school actually is doing.
These new systems, often called “data dashboards,” function the way a car’s dashboard does—by displaying multiple measures that affect how a school is performing.
Rothamn uses the term "dashboard" to refer to both a broader set of indicators beyond student assessment results (e.g. climate information) and a visual display of those data. Rothman does not distinguish between new accountability systems and display (aka dashboards). In fact, under the section Why Dashboards? Rothman says that "current accountability systems" have problems, "dashboards can alleviate some of the problems." By contrasting "current accountability systems" with "dashboards" Rothman has put two separate concepts into one idea. This causes confusion.
So, what do these terms mean. First, as Morgan Polikoff clearly lays out (Ed Researcher 2013), there are two primary streams of thinking that support accountability in education. The first stream of thinking is that the accountability system creates incentives that focus attention on behaviors that matter and result in student outcomes. The second stream of thinking is the educational consumer approach which broadly states that better information about school performance means better choices by consumers (parents and children). Thus, an accountability system is designed to facilitate accountability to someone, an organization, or a group. It includes measures that allow decision making (e.g. priority school, school in need of improvement). During the NCLB period the primary accountability system deployed by state agencies was AYP, which mostly depended on performance on state tests (in some cases growth was included). Some states had additional accountability systems that included measures beyond test scores. For example, Colorado included graduation and dropout rates in their School Performance Framework (SPF).
Second, according to Stephen Few, dashboards are a "visual display of the most important information needed to achieve one or more objectives which fits entirely on a single computer screen so it can be monitored at a glance." Few's definition includes four features of dashboards: (1) visual displays, (2) most important information needed to achieve an objective, (3) fit on a single screen, and (4) monitored at a glance.
In short, "dashboards" (or "data dashboards") will not solve our accountability problem. Dashboards are only a visual display. We must first design new accountability systems that meet the criteria set out by Morgan Polikoff. We might design a dashboard to display accountability information, but this is a separate effort after the new accountability system has been designed.
What Rothman may be trying to get at is that we need to improve our performance management focus. I described in more detail performance management in a previous blog post. In short, what Rothman might be arguing for is identifying leading and lagging indicators and developing technology that allows for ongoing monitoring of these indicators. Unfortunately, the mixing of terms that have specific meaning only confuses the reader.