RDP 2020-07: How Many Jobs Did JobKeeper Keep? Appendix D: Robustness Checks

Like any analysis using difference-in-differences, our results are sensitive to violations of the parallel trends assumption. As discussed in Section 5.2, the main way we address this assumption is to focus on fairly narrow tenure windows around the 12-month cut-off. In this appendix, we discuss the results of several robustness tests that address other concerns about violations of this assumption.

D.1 Pre-trends

Our first robustness test is to examine the employment trends in the treatment and control group leading up to the JobKeeper program. These trends for the period November 2019 to February 2020 are shown in Figure 2. We find that the employment trends were similar in the lead up to the program, with the 95 per cent confidence intervals for the difference-in-differences estimates for these periods spanning zero. This gives more confidence that the parallel trends assumption holds.

In saying that, in our set-up the pre-trends provide a weaker test of the parallel trends assumption than in some other contexts, because the treatment and control groups are restricted to people employed in February 2020 with at least six months of job tenure. As such, employment trends prior to February primarily reflect nuances in the definition of employment (see Section 5.5) rather than job creation and job destruction. For example, the 7½ per cent of people in the treatment and control groups who were not employed in January (Figure 2) will reflect those away from their job on unpaid leave for 4 weeks or longer over the summer holiday period, who are classified as not ‘employed’.

D.2 Controls

Our second robustness test is to examine whether our baseline results are robust to adding controls. Controls include dummies for industry of employment, occupational skill level, sex, migrant status, student status, multiple-job holding status, and a quadratic in age. All controls are measured in the pre-treatment period (February 2020), but are allowed to affect subsequent employment outcomes. For example, working in the accommodation & food industry in February is allowed to have an effect on a person's employment status in June and therefore accounts for the lingering effect of the lockdowns and social distancing on cafés. Adding these controls has no material effect on our results, which provides further confidence that parallel trends holds. This finding is not too surprising given our earlier finding that the treatment and control groups are balanced on observable factors (Table 1).

D.3 Placebo Tests Using Data from Earlier Periods

Our third robustness test is designed to tease out whether our baseline results are driven by higher turnover for the shorter-tenure casuals relative to longer-tenure ones, either in general or as part of a seasonal pattern. To do this, we look for evidence of a ‘placebo effect’ in an earlier period – here, the corresponding period of 2019. There is no evidence of a placebo effect; in 2019 employment losses of longer-tenure casuals were similar to shorter-tenure casuals (Figure D1, middle panel). In other words, underlying differences in turnover rates are not driving our results.

Unfortunately, the variable for the employee's status in employment (i.e. whether casual, permanent or self-employed) is only available in the LLFS from August 2014 onwards, which means we cannot not do a similar placebo test for the last major downturn – namely, the GFC in 2008–09.

D.4 Tenure Gradient in Employment Losses for Short-term Casuals

We also examined whether our baseline results are driven by last-in-first-out (LIFO) practices some firms may have used to prioritise dismissals during COVID-19. Specifically, longer-tenure casuals may have been more likely to retain their job than shorter-tenure casuals in the absence of JobKeeper, if firms tend to be more likely to dismiss their shorter-serving staff before their longer-serving staff during a downturn. Our baseline approach tries to circumvent this issue by restricting the estimation sample to a narrow range of tenure around the 12-month threshold. But for efficiency and data reasons, we still needed to include some people with modestly more tenure than others in our sample. The presence of LIFO practices may lead us to overstate the effect of JobKeeper on employment.

We test for this potential bias by examining whether short-term casuals with very short tenures (e.g. 1 or 2 months) in February 2020 were more likely to lose employment over the May to July period than short-term casuals with longer tenures (e.g. 9 or 10 months). To do this, we regress employment in a given month (say, July 2020) on the worker's tenure in their main job (a continuous variable) in February 2020. We do this for a sample of workers who were employed on a casual basis with between 1 and 10 months of tenure in February. All people in this group were ineligible for JobKeeper, so a positive and statistically significant ‘tenure gradient’ in employment would be evidence of LIFO practices. We find no statistically significant tenure gradient in May, June or July; indeed, in June and July the coefficient on the tenure variable has a negative sign, suggesting that, if anything, workers with more tenure were slightly less likely to remain employed than those with less tenure.^[53]

D.5 Placebo Tests Using Non-casual Employees

We can also test for the presence of LIFO practices by looking at how employment rates of non-casual workers changed around the same tenure level as in our baseline approach. This is a useful exercise because non-casual workers are not subject to the 12-month tenure rule under JobKeeper, which means that any divergence in employment rates of non-casual workers around this threshold must be due to something other than the effects of JobKeeper, such as LIFO practices. We do not see any meaningful differences in the employment rates for non-casual employees with 6–10 months of tenure (in February 2020) relative to those with 12–23 months of tenure (Figure D1, right-hand panel). We take this as further evidence that any LIFO practices are not driving our baseline results.

The results of this placebo test also shed light on the role of firing costs, which can influence a firm's decisions to dismiss an employee when conditions deteriorate. In Australia, permanent employees are eligible for redundancy pay (a key component of firing costs) after 12 months of continuous service at a firm (FWO 2020c).^[54] This means our placebo test using non-casual workers can be interpreted as a test both for LIFO effects and/or for the effects of higher firing costs on worker turnover (both effects are expected to operate in the same direction). As such, another interpretation of the placebo test result is that differences in firing costs are not important for explaining differences in employment outcomes across workers during the first few months of the COVID-19 crisis. Permanent employees eligible for redundancy pay did not have lower rates of job loss than those without it (Figure D1). However, this interpretation is muddied somewhat because the 12-month threshold for redundancy pay-eligibility is not anchored at a fixed point in time (unlike the threshold for JobKeeper eligibility which was anchored at 1 March), which means that some of the lower-tenure group became eligible for redundancy pay during the April to July 2020 period.

D.6 Excluding Multiple-job Holders

Our final robustness test is to examine if our results are sensitive to excluding multiple-job holders from the estimation sample. As discussed in Section 5.5, some individuals in our sample held more than one job prior to JobKeeper, but each person could only receive JobKeeper from their primary employer. Around 9 per cent of our treatment and control groups held more than one job in February 2020 (Table 1). In our baseline approach, we assigned individuals to the treatment and control groups based on the characteristics of their ‘main job’ in February.^[55] There is not enough information in the LLFS on an individual's secondary job(s) to discern if those jobs were also worker-eligible for JobKeeper.

Our baseline estimates will understate the effect of JobKeeper worker eligibility on employment if some individuals in our control group were eligible for JobKeeper via their secondary job(s). One way to explore the extent of this bias is to re-estimate our models after excluding those who held more than one job prior to JobKeeper. This yields estimates of the effect of JobKeeper worker eligibility on employment that are 0.9 to 1.9 percentage points (or 10 to 20 per cent) larger than our baseline estimates during April to July. While this may suggest that some multiple-job holders in our control group did receive JobKeeper (and so our baseline estimates are biased downward), it may also reflect that multiple-job holders were less likely to have exited employment in the absence of JobKeeper.^[56] In the LFS, a person remains ‘employed’ if they hold onto at least one of their jobs; all else being equal (and assuming that job losses are independent events), this is more likely to occur if a person had more jobs to begin with.^[57] In other words, the treatment effect of JobKeeper on employment is plausibly smaller for multiple-job holders than for single-job holders because the former group were more likely to retain at least one job in the absence of JobKeeper. Again, it is worth reiterating that our analysis focuses on whether JobKeeper preserved employment, not jobs.

Footnotes

The p-values on the tenure coefficient in May, June and July are 0.14, 0.86 and 0.62, respectively. [53]

After one year, the employee is entitled to four weeks of pay at their base pay rate in the event of being made redundant. The generosity of these redundancy payments increases with every additional year of service up to ten years (FWO 2020c). [54]

In the LFS, an individual's main job is the job in which they usually work the most hours. [55]

Part of the difference might also reflect that multiple-job holders in our treatment group were more likely to receive JobKeeper than single-job holders in our treatment group, to the extent that they were also worker-eligible in their secondary jobs. [56]

Our analysis of ABS data for 2016/17 (ABS 2019b) suggests that two-thirds of multiple job holders who worked in one of the industries that were most adversely affected by COVID-19 (i.e. hospitality, arts, retail, real estate or other services) were partly ‘insured’ against the shock as they also held a second job in one of the less-affected industries. [57]