Data-Driven Policymaking? In Your Dreams

Gathering data to make informed decisions is one thing, using it is another.  With computing power so inexpensive and accessible, with so much public and private sector pressure to collect information, practitioners and policymakers have at their fingertips real-time data on everything important. Consider the economy.

Writing in the New Yorker, James Surowiecki (see below) points out that

”[O]ur picture of the economy is more detailed and sophisticated than ever, and that makes it easier for businesses and the government to react quickly to changes in the economy.”

Surowiecki describes the Billion Prices Project designed by two economists that identifies Web-based prices for online goods—more than 500,000 daily or five times the number that  federal agencies gather—to tell what is happening now with prices, not a month later as U.S. data do. For those who worry about inflation, the Billion Prices Project noted businesses cutting prices months before government numbers did. Such a project, then, can help keep governments honest when they make economic policy decisions. Why honest? It is no secret that governments around the world “manipulate economic data—downplaying inflation [and] overstating job growth.”

But there is a problem even when real-time information is available. “Giving policymakers more information doesn’t mean that they’ll believe it or act on it,” Surowiecki says. Amen.

Surowiecki  points to the 2008 financial crisis when housing prices spiked astronomically and banks shoveled loans to anyone with a pulse rate. Yet  bank regulators and Federal Reserve officials with much data in hand from earlier years, took no action. The bubble burst.  Current talk among the same officials about tightening up monetary policy mirrors their fears of inflation yet available data (e.g., Consumer Price Index, the Billion Prices Project) counter those fears.

Please don’t shake your head in dismay at how ideology and emotions–note above policymaker fears and remember greed–get in the way of sensible decision-making. In the midst of school reform, policymakers are flush with data on important decisions yet either fail to act or forge ahead with policies that  ignore data.

I do not argue that all decisions must be driven by current data or even what experts say. If that were the case, we would be in the eerie, hyper-rational world of Hal the omniscient computer from 2001: Space Odyssey. Most major decisions with public consequences (e.g., recalling autos; evaluating teachers on the basis of student test scores), involve incomplete and conflicting data. And values.  Judgments have to be made. Trade-offs among rival values have to be  satisficed. No amount of data can point to the right judgment.

I do argue, however, that when sufficient data are available about a major policy decision that contradicts or challenges an option under serious consideration, policymakers have an obligation to seek public discussion, make clear–again publicly–why they disregard relevant data in making their decision, and most important, be explicit if other reasons account for the decision. That has not happened with the Common Core Standards which is hardly  grounded in sufficient evidence in preparing a national curriculum–see Christopher Tienken’s piece in Journal of Scholarship and Practice, Winter2011.FINAL .

Nor has it occurred with the recent New York state Board of Regents’ decision to have 40 percent of a teacher’s evaluation determined by students’ test scores. The background to this decision is that New York won $700 million  in the Obama administration’s Race to the Top competition after legislating new rules for charters and  establishing procedures to determine teacher and principal effectiveness based on student test scores. New York state, after having cut education funds, needs that chunk of federal money.

A virtual Who’s Who of education measurement specialists wrote to the Board of Regents challenging the Commissioner’s recommendation to approve these procedures for evaluating teachers. Citing research reports from RAND and The National Research Council of the National Academy of Sciences including reviews of the literature on using test scores or “value added measures” (VAM), the letter listed the dangers of judging a teacher’s effectiveness based on test scores. “We urge you, ” they concluded,  “to reject proposals that would place significant emphasis on this untested strategy that could have serious negative consequences for teacher and for the most vulnerable students in the State’s schools”(Regents-Teacher-Evaluation-Letter-5.15.11).

The Chancellor of the Board of Regents said: “These are not perfect tools by any means. But that being said, I believe  it is important to have an objective system to evaluate teachers on a professional basis. This is the beginning of such a process.”

Giving policymakers the latest data and advice from top experts in a field does not mean they will use it.

By the way, I could not find any mention of New York’s competing for Race to the Top money and enhancing their quest for scarce dollars.

Related articles

• James Surowiecki: The value of economic data. (newyorker.com)