RDP 2020-07: How Many Jobs Did JobKeeper Keep? 7. Robustness and Potential Biases

7.1 The Parallel Trends Assumption

The validity of our difference-in-differences approach rests on the parallel trends assumption. The assumption is that the change in the employment rate of the treatment group would have been the same as the control group in the absence of JobKeeper. The predominant way this is addressed in the literature this is to focus on fairly tight tenure windows around the 12-month tenure cut-off. This is done because employees who narrowly missed out on eligibility should be similar to those who only narrowly qualified, and therefore absent JobKeeper these two groups are likely to have moved in a similar fashion. However, because we do allow for a modest range of tenure around the threshold it is possible this assumption is violated. For example, the parallel trends assumption may not hold if:

• firms used ‘last-in-first-out’ methods to prioritise redundancies during COVID-19,
• social distancing and other restrictions had a different effect on the treatment group than the control, and/or
• shorter-term casuals ordinarily have a higher rate of job turnover than longer-term casuals.

In Appendix D we present the results of several robustness tests that are designed to address these potential violations of the parallel trends assumption. In all cases, we do not find any evidence to suggest that the parallel trends assumption is violated. The robustness tests include: (i) examining the pre-trends in the employment rates, (ii) adding a rich set of pre-treatment controls to Equation (2), (iii) looking for evidence of placebo effects in prior years, (iv) looking for evidence of a tenure gradient in employment losses for short-term casuals, and (v) looking for evidence of placebo effects in a sample of non-casual (e.g. permanent) employees around the 12-month tenure cut-off.

7.2 Spillovers to the Control Group

One source of bias that we were unable to test for is the possibility that JobKeeper had spillover effects on the control group. On the one hand, JobKeeper reduced firms' after-subsidy labour costs, which may have enabled them to retain or hire more ineligible staff than they would have done in the absence of the subsidy. There are some reports of this occurring.^[34] All else being equal, this spillover would lead us to underestimate the effect of JobKeeper on employment, because some of the workers we allocate to the control group also benefited from JobKeeper.^[35]

On the other hand, the control group might have been adversely affected by JobKeeper to the extent the wage subsidy reduced the price of retaining eligible employees relative to ineligible employees at a firm. This ‘substitution effect’ would lead us to overestimate the effect of JobKeeper, all else being equal.

Different readers are likely to have different views on which of these two effects dominate in practice (if any). For example, some readers may argue that the income effects would outweigh the substitution effects because firms were constrained in substituting towards more eligible workers. Indeed, there was a limit on the amount of substitution that could occur under the program given that worker eligibility was based on hiring decisions in early 2019 and therefore predetermined. If these constraints on substitution were binding and if income effects were large, our estimates are likely to understate the effects of JobKeeper on employment, all else being equal. A counterargument is that few firms were constrained on the substitution margin, since eligibility required a sharp decline in revenue which presumably would have led firms to dismiss a large number of eligible and ineligible staff in the absence of JobKeeper.^[36] In that case, firms could ‘increase’ their use of eligible staff by simply retaining more of those staff than they would have without JobKeeper, and fall short of the point where the constraint on the substitution effect binds.

Without having access to linked employee-employer data we have no way of quantifying the relative importance of these two offsetting sources of bias. In presenting our estimate we assume that the negative and positive biases balance out. This is a key maintained assumption in our analysis and an important caveat to our overall estimate of JobKeeper's effect. Future research using alternative datasets should attempt to explore the validity of this assumption in more detail.

7.3 Alternative Classifications of Employment

The question we are ultimately interested in is the extent to which JobKeeper preserved worker-firm relationships. To do this, we have focused on whether JobKeeper cushioned the decline in ‘employment’, which is often taken as a proxy for worker-firm relationships. However, employment is not necessarily the best proxy.

The COVID-19 crisis has brought into focus some important nuances in how the ABS classifies employment, and how this treatment differs to that used in other countries. As discussed in Section 5.5, in Australia a person who is stood down is classified as employed for the first four weeks, and thereafter is classified as either employed or not employed depending on whether they are paid by their employer.^[37] In May, 437,000 people (2.1 per cent of the working-age population) were classified as not being employed even though they had a job to return to (Figure 3). This number fell to 288,000 in June, and 209,000 in July.

Because we are interested in attachment to a firm, we also consider a measure of employment that is broader than the ABS measure. This broader measure includes both the officially ‘employed’ and those who are not classified as employed by the ABS, but report being away from a job and, thus, appear to remain connected to an employer. Our estimates using this broader measure are similar to those using the standard ABS measure of employment (Figure 4). This provides further support for our overall assessment that JobKeeper has been important in preserving worker-firm relationships over and above what would have taken place without the program.

We also consider a narrower measure of employment that re-classifies workers who are stood down (or are away from work for any reason) as not employed. This is similar to the treatment of temporary lay-offs in US and Canadian labour force surveys (see ABS (2020b) for details). Our regression estimates using this measure are also similar to those using the standard ABS measure of employment (Figure 4). One interpretation of this result is that most employees on JobKeeper in May and June were doing productive work for their firm, rather than stood down. This is consistent with analysis by Treasury (2020b) that finds that only a small share of JobKeeper recipients were away from work during this period.^[38]

The key takeaway from this section is that our findings are not driven by some nuance in the way the ABS classifies employment.

7.4 Alternative Identification Strategy

Our results are also robust to using a completely different identification strategy. In this alternative approach we focus on differences in worker eligibility arising from the residency requirement, rather than the 12-month rule. With the exception of New Zealand citizens, temporary residents were not eligible to receive JobKeeper. The details of this approach are described in Appendix E, but, in short, it involves comparing the employment outcomes of temporary migrants from New Zealand (potentially eligible for JobKeeper) to that of temporary migrants from other countries (ineligible for JobKeeper). Our estimates of the effects of JobKeeper on employment are larger using the residency-based approach than with the tenure-based approach. However, for reasons we discuss in Appendix E, our preferred estimates are those based on the 12-month tenure rule for casuals. We mainly use this alternative approach as a sense check on our overall conclusions from our baseline estimates.

7.5 Hours Worked

Although our interest is mainly in the effects of JobKeeper on employment, it is useful to consider whether the program also had a causal effect on hours worked. To do this, we replaced our binary measure of employment in Equation (2) with a continuous measure of the change in hours worked since February 2020. In cases where a person became unemployed or exited the labour force after February, we set their hours to zero in that month, rather than dropping them from the sample. For this reason, our estimates in this section correspond to the effect of JobKeeper worker eligibility on total hours worked, rather than average hours.

The treatment and control groups both worked considerably fewer hours over the April to July period relative to February (Figure 5). This fall in total hours worked was due to both a decline in employment and a fall in average hours worked. Taken at face value, the mean differences between the groups implies that JobKeeper boosted total hours by around 1 hour per week across the April to June period. This estimate – which is equivalent to 5 per cent of total hours worked in February – is broadly similar to our estimate of the effect on employment. In other words, the point estimates suggest that the effects of JobKeeper mainly operated through the extensive margin (employment) rather than the intensive margin (average hours).^[39] This finding would be consistent with recent analysis of the Paycheck Protection Program in the United States, which found the scheme had an effect on employment, but not average hours (Autor et al 2020). In saying that, our estimates for hours are not statistically significant, which may reflect greater noise in the hours data. Also, there is some evidence that JobKeeper had an ‘effect’ on hours in March, which pre-dates JobKeeper. With these considerations in mind, our estimates for hours should be treated with caution.

There are some theoretical reasons to expect that JobKeeper may not have had a positive effect on average hours. The flat $1,500 payment means the marginal cost of increasing an employee's hours is zero up to a point, after which it equals their hourly wage. For this reason, some firms had an incentive to redistribute hours amongst their staff, and retain some workers they otherwise would have let go. It is possible that these adjustments had a neutral net effect on average hours worked. In addition, some lower-earning employees may have been unwilling to increase their hours given that the marginal benefit to them from doing so (in terms of their earnings) was also zero up to a point.

Footnotes

See the business case study on page 28 of Treasury (2020b). [34]

Another consideration is the effects of cross-subsidisation coming from infra-marginal employees at the firm who received JobKeeper (e.g. eligible casual employees at the firm with more than two years of job tenure or eligible permanent employees). However, there is no clear reason to believe that this cross-subsidisation effect should be any larger for our control group than for our treatment group. [35]

When thinking about the substitution effect of the JobKeeper Payment program, we abstract from the income effect of the JobKeeper Payment program itself but not the adverse shock to firm incomes caused by the COVID-19 crisis. [36]

The latter group comprises those who have been stood down for more than four weeks without pay. [37]

Treasury (2020b) analysis of business micro data suggests that in the fortnight ending 24 May 400,000 people were stood down on JobKeeper, or around 11½ per cent of all recipients. This is consistent with our analysis of the LLFS micro data, which suggests that in May around 310,000 people were classified as ‘away from work’ for more than four weeks but still being paid by their employer. The small discrepancy between the Treasury's estimate and our own estimate based on the LLFS may reflect that our estimate excludes any stood down JobKeeper recipients who also held a second job or who found work at another firm. [38]

If the effect of JobKeeper worker eligibility on employment and total hours is 7 per cent and 5 per cent respectively, the effect on average hours worked is approximately –2 per cent. [39]