RDP 2017-01: Gauging the Uncertainty of the Economic Outlook Using Historical Forecasting Errors: The Federal Reserve's Approach 1. Introduction

Since late 2007, the Federal Open Market Committee (FOMC) of the US Federal Reserve has regularly published assessments of the uncertainty associated with the projections of key macroeconomic variables made by individual Committee participants.^[1] These assessments, which are reported in the Summary of Economic Projections (SEP) that accompanies the FOMC minutes once a quarter, provide two types of information about forecast uncertainty. The first is qualitative in nature and summarizes the answers of participants to two questions: Is the uncertainty associated with his or her own projections of real activity and inflation higher, lower or about the same as the historical average? And are the risks to his or her own projections weighted to the upside, broadly balanced, or weighted to the downside? The second type of information is quantitative and provides the historical basis for answering the first qualitative question. Specifically, the SEP reports the root mean squared errors (RMSEs) of real-time forecasts over the past 20 years made by a group of leading private and public sector forecasters.

We begin this paper by discussing the motivation for central banks to publish estimates of the uncertainty of the economic outlook, and the advantages – particularly for the FOMC – of basing these estimates on historical forecast errors rather than model simulations or subjective assessments. We then describe the methodology currently used in the SEP to construct estimates of the historical accuracy of forecasts of real activity and inflation, as well as extending it to include uncertainty estimates for the federal funds rate. As detailed below, these estimates are based on the past predictions of a range of forecasters, including the FOMC participants, the staff of the Federal Reserve Board, the Congressional Budget Office, the Administration, the Blue Chip consensus forecasts, and the Survey of Professional Forecasters.^[2] After that, we review some of the key properties of these prediction errors and how estimates of these properties have changed in the wake of the Great Recession. We conclude with a discussion of how this information can be used to construct confidence intervals for the FOMC's SEP forecasts – a question that involves grappling with issues such as biases in past forecasts and potential asymmetries in the distribution of future outcomes.

Several conclusions stand out from this analysis. First, differences in average predictive performance across forecasters are quite small. Thus, errors made by other forecasters on average can be assumed to be representative of those that might be made by the FOMC. Second, if past forecasting errors are any guide to future ones, uncertainty about the economic outlook is quite large. Third, error-based estimates of uncertainty are sensitive to the sample period. And finally, historical prediction errors appear broadly consistent with the following assumptions for constructing fan charts for the FOMC's forecasts: median FOMC forecasts are unbiased, intervals equal to the median forecasts plus or minus historical RMSEs at different horizons cover approximately 70 percent of possible outcomes, and future errors that fall outside the intervals are distributed symmetrically above and below the intervals. That said, the power of our statistical tests for assessing the consistency of these three assumptions with the historical data is probably not great. In addition, the effective lower bound on the level of the nominal federal funds rate implies the distribution of possible outcomes for short-term interest rates should be importantly asymmetric in a low interest-rate environment.

Footnotes

The Federal Open Market Committee consists of the members of the Board of Governors of the Federal Reserve System, the president of the Federal Reserve Bank of New York, and, on a rotating basis, four of the remaining eleven presidents of the regional Reserve Banks. In this paper, the phrase ‘FOMC participants’ encompasses the members of the Board and all twelve Reserve Bank presidents because all participate fully in FOMC discussions and all provide individual forecasts; the Monetary Policy Report to the Congress and the Summary of Economic Projections provide summary statistics for their nineteen projections. [1]

This discussion updates and extends the overview provided by Reifschneider and Tulip (2007) and Federal Reserve Board (2014). [2]