How the CBO gained its reputation

From Alice Rivlin:

The most important decision in CBO’s formative days, Rivlin told the hundreds of staff and alumni at the Capitol Visitor Center, “was not to make recommendations in reports. That allowed me and my successors to defend the CBO for 40 years” against attacks from lawmakers upset with a score on the cost of a bill and a “skeptical press that did its best to trip us up.”

There’s more in GovExec.

Pay-for-Performance and the Senior Executive Service

GAO compared SES members’ performance awards against their evaluation ratings in the 24 largest agencies. It found nearly all SES members rated better than average, a statistic improbability anywhere but Lake Wobegone ofA Prairie Home Companion fame. About 85 percent of SES members scored either a 5, the highest rating, or a 4, the second highest, for fiscal 2010-2013.

Supervisors feel pressure to rate executives highly because the ratings become criteria for myriad compensation decisions, GAO said.

But budget constraints and recent guidance from the Office of Management and Budget has kept the total amount of performance awards down and erased distinctions among SES. Executives with ratings of 5 earned $4,991 more than those with ratings of 4 in fiscal 2010. By 2013, the difference had shrunk to just $2,604. Between those in the middle of the pack, the difference is even smaller. Executives with ratings of 4 took home $690 more than those with ratings of 3.

As the difference lessens, the perception that awards are not directly linked to performance grows, GAO noted. Yet an executive at rating level 5 is much more likely to receive a performance award than one rated at 4 or below.

More at Federal News Radio.

An accessible introduction to the problem of multiple testing

From Cam Harvey, with applications to portfolio management:

We provide some new tools to evaluate trading strategies. When it is known that many strategies and combinations of strategies have been tried, we need to adjust our evaluation method for these multiple tests. Sharpe Ratios and other statistics will be overstated. Our methods are simple to implement and allow for the real-time evaluation of candidate trading strategies.

Recommended. This is a huge issue in classifying causal mechanisms in medicine, evidence-based policymaking and management, etc.

On one hand, any given “experiment” (read: regression result) may be faulty (sample-dependent); on the other hand, if we have multiple tests, we need good ways of aggregating those results.

If we have this problem with inference in the field of finance (with the amount of time, attention, and cash expended to figure out the best trading strategies), then what hope is there in evidence-based policymaking (where, in most cases, we have – at best – one experiment)?

Statistics, strong statements, and evidence-based policy

From a discussion of the new dietary guidelines:

Instead of accepting that this evidence was inadequate to give sound advice, strong-willed scientists overstated the significance of their studies.

Much of the epidemiological data underpinning the government’s dietary advice comes from studies run by Harvard’s school of public health. In 2011, directors of the National Institute of Statistical Sciences analyzed many of Harvard’s most important findings and found that they could not be reproduced in clinical trials.

Source: NYT.

Data: Regulatory Prosecutions Reach a New Low

From TRAC:

The latest available data from the Justice Department show that during December 2014 the government reported 90 new government regulatory prosecutions, the lowest count for a single month in this program category since 1998. According to the case-by-case information analyzed by the Transactional Records Access Clearinghouse (TRAC), this number is down 41.2 percent over the previous month.

When monthly 2014 prosecutions of this type are compared with those of the same period in the previous year, the number of filings was down (-14.3%). Prosecutions over the past year are still much lower than they were five years ago. Overall, the data show that prosecutions of this type are down 30.5 percent from levels reported in 2009.

Developing Knowledge States: Technology and the Enhancement of National Statistical Capacity

Now available on SSRN, with Derrick Anderson of Arizona State University:

National statistical systems are the enterprises tasked with collecting, validating and reporting societal attributes. These data serve many purposes – they allow governments to improve services, economic actors to traverse markets, and academics to assess social theories. National statistical systems vary in quality, especially in developing countries. This study examines determinants of national statistical capacity in developing countries, focusing on the impact of general purpose technologies (GPTs). Just as technological progress helps to explain differences in economic growth, states with markets with greater technological attainment (specifically, general purpose technologies) arguably have greater capacity for gathering and processing quality data. Analysis using panel methods shows a strong, statistically significant positive linear relationship between GPTs and national statistical capacity. There is no evidence to support a non-linear function in this relationship. Which is to say, there does not appear to be a marginal depreciating National Statistical Capacity benefit associated with increases in GPTs.

Good news for users of federal government data

From Government Executive:

In a victory for transparency advocates, the Office of Management and Budget on Friday acceded to a Freedom of Information Act request that it release for the first time a centralized set of inventories of the data sets kept by each federal agency.

The Sunlight Foundation had enlisted a law firm to pry loose agency indexes that will be complete this spring. The inventories will benefit information technology managers, FOIA requesters and agency oversight efforts, according to the nonprofit.

The inventories — which OMB will release in March — are being created in response to President Obama’s May 2013 executive order pushing circulation of government data “to promote continued job growth, government efficiency, and the social good that can be gained from opening government data to the public.”

OMB’s FOIA office originally told the Sunlight requesters they would have to go to each of 24 major agencies separately. But after a refiling from the Garvey Schubert Barer law firm, OMB on Feb. 6 sent a letter promising to release the material in March.

In a Monday blogpost, Sunlight staffers wrote that “for the first time, the United States government has agreed to release what we believe to be the largest index of government data in the world.”