RDP 2023-07: Identification and Inference under Narrative Restrictions 1. Introduction

Understanding the dynamic causal effects of structural shocks is one of the central problems in macroeconometrics, and there is increasing empirical demand for methods that require minimal identifying assumptions. Replacing point-identifying restrictions in structural vector autoregressions (SVARs) with set-identifying sign restrictions is an early example of this search for robustness (e.g. Uhlig 2005). A more recent example is the idea of substituting or augmenting traditional restrictions on structural parameters with ‘narrative restrictions’ (henceforth NR), which are inequalities involving structural shocks in given time periods (Antolín-Díaz and Rubio-Ramírez (2018) (henceforth AR18); Ludvigson, Ma and Ng (2018)). These restrictions force the SVAR's predictions to be consistent with narratives about the nature of structural shocks driving macroeconomic variation in particular historical episodes. The promise of these restrictions is that they may deliver informative inferences about the effects of structural shocks under weak or uncontroversial restrictions on the structural parameters.

An example of NR are ‘shock-sign restrictions’, such as the restriction in AR18 that the US economy was hit by a positive monetary policy shock in October 1979. This is when the Federal Reserve increased the federal funds rate following Paul Volcker becoming chairman, and is widely considered an example of a positive monetary policy shock (e.g. Romer and Romer 1989). AR18 also consider ‘historical decomposition restrictions’, such as the restriction that the change in the federal funds rate in October 1979 was overwhelmingly due to a monetary policy shock. Other restrictions also fit within this framework, including restrictions on shock magnitudes (e.g. Ludvigson et al 2018) or rankings (e.g. Ben Zeev 2018).

A burgeoning literature imposes NR in a broad range of empirical applications.^[1] However, the non-standard nature of these restrictions raises econometric challenges. Under these restrictions, there are no formal results on identification or the validity of frequentist approaches to inference.^[2] Moreover, as we show in this paper, the Bayesian procedure of AR18, which is used by the majority of the literature, may be sensitive to prior choice. This paper contributes to the literature by formally analysing identification and inference in models with NR, and by providing an approach to inference that eliminates prior sensitivity. Importantly, this approach is valid from both Bayesian and frequentist perspectives.

From a frequentist perspective, NR are fundamentally different from traditional restrictions. Under normally distributed structural shocks, traditional sign restrictions induce set identification, because they generate a set-valued mapping from the SVAR's reduced-form parameters to its structural parameters – an identified set – that represents observational equivalence. The identified set corresponds to the flat region of the likelihood and, by the definition of observational equivalence, does not depend on the realisation of the data (e.g. Rothenberg 1971). NR also result in the likelihood possessing flat regions and hence generate a set-valued mapping from the reduced-form parameters to the structural parameters. Crucially, this mapping additionally depends on the realisation of the data. The data dependence of this mapping implies that the standard concept of an identified set does not apply. In turn, this means that: 1) existing results on identification in SVARs are inapplicable; and 2) there is no known valid frequentist procedure for inference.

From a Bayesian perspective, the bulk of the empirical literature conducts Bayesian inference under NR in a similar way as under traditional restrictions by following a procedure in AR18. We highlight two issues with the existing approach: the potentially spurious effects on inference of using a conditional likelihood to construct the posterior; and the sensitivity of inference to prior choice due to the likelihood possessing flat regions. The prior sensitivity of the existing Bayesian approach makes it difficult to know whether apparently informative inference obtained in empirical studies (e.g. narrow credible intervals) reflects the informativeness of NR or the choice of prior. Removing the effect of the prior allows us to understand if NR deliver on their promise of offering informative inference under minimal assumptions, in contrast with traditional sign restrictions, which have been shown to provide little information in some settings (e.g. Baumeister and Hamilton 2015; Wolf 2020; Read 2022b).

The paper proceeds in four main steps. First, we formalise the identification problem under NR. Second, we propose using the unconditional likelihood, rather than the conditional likelihood, to construct the posterior. Third, we consider a robust (multiple-prior) Bayesian approach to assess and/or eliminate the posterior sensitivity that remains when using the unconditional likelihood. Finally, we show that the robust Bayesian approach has frequentist validity in large samples.

To the best of our knowledge, this is the first paper to study identification under general NR. Plagborg-Møller and Wolf (2021b) note that shock-sign restrictions could in principle be cast as an external instrument (or ‘narrative proxy’) and used to point identify impulse responses in a local projection. Plagborg-Møller (2022) argues that such an approach possesses several appealing robustness properties relative to the likelihood-based approach of AR18 that we consider here, including that it allows for imperfect narrative information and non-invertibility.^[3] Petterson, Seim and Shapiro (2023) derive bounds for a slope parameter in a single equation given restrictions on the magnitude of the residuals, but the setting is non-probabilistic.

We make two main contributions to the understanding of identification under NR. First, we provide a necessary and sufficient condition for global identification of a SVAR under NR and as an example show that this condition is satisfied in a bivariate SVAR with a single shock-sign restriction. This means that, in contrast with traditional sign restrictions, NR may be formally point identifying despite generating a set-valued mapping from reduced-form to structural parameters in any particular sample. This result does not, however, deliver a point estimator, because the observed likelihood is almost always flat at the maximum. Second, we introduce the notion of a ‘conditional identified set’, which extends the standard notion of an identified set to a setting where identification is defined in a repeated sampling experiment conditional on the observations entering the NR. This provides an interpretation for the set-valued mapping induced by the NR as the set of observationally equivalent structural parameters in such a conditional frequentist experiment. We make use of the conditional identified set when analysing the frequentist properties of our procedure.

The fact that NR deliver a set of maximum likelihood estimators is reminiscent of maximum score estimation, where the objective function yields a set of maximisers (Manski 1975, 1985). Our conditional identified set, which fixes the flat regions of the likelihood in the conditional frequentist experiment, shares geometric properties with the finite sample identified set introduced by Rosen and Ura (2020) in the maximum score context; however, their finite sample inference procedure does not apply here.

In terms of inference under NR, our contribution can be viewed from both a Bayesian and a frequentist point of view. Our first message to Bayesian researchers is to base analysis on the unconditional likelihood, rather than the conditional likelihood used by AR18. Conditioning can be problematic because, for some types of NR, a component of the prior is updated only in the direction that makes the NR unlikely to hold ex ante. This is due to conditioning on a non-ancillary event, which results in loss of information.

Our second message to Bayesian researchers is that posterior inference may be sensitive to the choice of prior, because the unconditional likelihood has flat regions under NR (the conditional likelihood also has flat regions under shock-sign restrictions). This sensitivity is a problem that also occurs in set-identified models under traditional restrictions (e.g. Poirier 1998; Baumeister and Hamilton 2015). As advocated for by Giacomini and Kitagawa (2021a) (henceforth GK21), this problem can be solved by adopting a robust (multiple-prior) Bayesian approach. GK21 consider robust Bayesian inference in SVARs under traditional set-identifying restrictions, a setting where – unlike in our case – frequentist inference is also available (e.g. Gafarov et al 2018; Granziera et al 2018). They decompose the prior for structural parameters into a prior for reduced-form parameters, which is revisable, and a conditional prior for structural parameters given reduced-form parameters, which is unrevisable. Considering the set of all conditional priors satisfying the identifying restrictions generates a set of posteriors. This removes the source of posterior sensitivity and makes robust Bayesian and frequentist approaches asymptotically equivalent, reconciling the disagreement between frequentist and Bayesian methods that arises in set-identified models (Moon and Schorfheide 2012).^[4]

We explain how this robust Bayesian approach can be adapted to NR. Even if a researcher has a credible prior, we recommend reporting the standard Bayesian posterior (under the unconditional likelihood) together with the robust Bayesian output. This allows researchers to assess the extent to which posterior inference may be driven by prior choice. In the absence of a credible prior, we recommend reporting the robust Bayesian output as an alternative to the standard Bayesian posterior.

This paper's contribution to frequentist inference is to provide the first (to our knowledge) asymptotically valid approach to inference under NR. While other frequentist approaches are in principle possible, one appealing feature of the robust Bayesian approach is its numerical tractability. Proving the frequentist asymptotic validity of the approach is challenging, due to the data-dependent mapping induced by the NR that we discussed above. This means that the results in GK21 about the asymptotic equivalence between Bayesian and frequentist inference are not applicable here. We address these challenges by deriving new results on the asymptotics of robust Bayesian analysis under a fixed number of NR, which we argue is the empirically relevant case given the small number of restrictions typically imposed in the literature. We show that, under regularity conditions, the robust credible region provides asymptotically valid frequentist coverage of the conditional identified set for the impulse response, which also implies correct coverage for the true impulse response.

We illustrate our methods by revisiting the monetary SVAR in AR18. We first examine the robustness of conclusions about the output effects of US monetary policy when NR are imposed based only on the Volcker episode. We find that inferences about the output response are sensitive to prior choice, and the restrictions are largely uninformative in the sense that they admit a wide range of positive and negative output responses. Restrictions based on the Volcker episode in isolation are therefore not sufficient to precisely identify the effects of monetary policy. We then impose an extended set of NR related to multiple episodes, and find robust evidence that output falls following a positive monetary policy shock. Disentangling the informativeness of the different restrictions, the shock-sign restrictions on their own are not particularly informative, and drawing robust conclusions about the output response relies on imposing restrictions on the historical decomposition.

The remainder of the paper is structured as follows. Section 2 highlights the econometric issues that arise when imposing NR using a bivariate example. Section 3 describes the general framework. Section 4 analyses global identification under NR and introduces the concept of a conditional identified set. Section 5 discusses how to conduct standard and robust Bayesian inference under NR. Section 6 explores the frequentist properties of the robust Bayesian approach. Section 7 contains the empirical application and Section 8 concludes. The appendices contain proofs and other supplemental material.

Notation: For the matrix X, vec(X) is the vectorisation of X and vech(X) is the half-vectorisation. e_i,n is the ith column of the n × n identity matrix, I_n. 0_n×m is a n × m matrix of zeros. 1 (.) is the indicator function.

Footnotes

Examples of papers imposing NR include Ben Zeev (2018), Altavilla, Darracq Pariès and Nicoletti (2019), Furlanetto and Robstad (2019), Cheng and Yang (2020), Kilian and Zhou (2020, 2022), Laumer (2020), Redl (2020), Zhou (2020), Antolín-Díaz, Petrella and Rubio-Ramírez (2021), Caggiano et al (2021), Larsen (2021), Ludvigson, Ma and Ng (2021), Maffei-Faccioli and Vella (2021), Berger, Richter and Wong (2022), Fanelli and Marsi (2022), Inoue and Kilian (2022), Badinger and Schiman (2023), Berthold (2023), Caggiano and Castelnuovo (2023), Conti, Nobili and Signoretti (2023), Harrison, Liu and Stewart (2023), Herwatz and Wang (2023), Neri (2023), Reichlin, Ricco and Tarbé (2023), Ascari et al (forthcoming), Boer, Pescatori and Stuermer (forthcoming) and Rüth and Van der Veken (forthcoming). [1]

Ludvigson et al (2018, 2021) use a bootstrap to conduct inference, but do not provide evidence about its validity. Existing frequentist approaches to conducting inference in set-identified SVARs include Gafarov, Meier and Montiel Olea (2018) and Granziera, Moon and Schorfheide (2018). [2]

Giacomini, Kitagawa and Read (2022a) explore the performance of the weak-instrument robust frequentist inferential procedures from Montiel Olea, Stock and Watson (2021) when using narrative proxies; these procedures may suffer from size distortions when the sign of the shock is known in a small number of periods. [3]

Giacomini, Kitagawa and Read (2022b) extend this approach to SVARs where the parameters of interest are set identified using external instruments, or ‘proxy SVARs’. See Giacomini, Kitagawa and Read (2021) for a survey of the literature on robust Bayesian methods, including a discussion of different approaches to conducting robust Bayesian inference in set-identified SVARs. [4]