RDP 2020-07: How Many Jobs Did JobKeeper Keep? 5. Empirical Strategy

The first step in our empirical strategy is to estimate the effect of JobKeeper worker eligibility on employment. We estimate this parameter using a difference-in-differences approach. This quasi-experimental approach allows us to control for unobserved variables that bias estimates of causal effects. In this case, we focus on two groups of workers that are similar except that one group passed the worker-eligibility test and may have received JobKeeper (the treatment group) while the other group did not pass the worker-eligibility test and was ineligible for JobKeeper (the control group).^[15] We argue that any differences in the employment rates of these two groups that emerged after the introduction of JobKeeper gives us an estimate of the causal effect of JobKeeper worker eligibility on employment. This differs from the effect of actually receiving JobKeeper on employment because not all worker-eligible individuals received JobKeeper (see Section 5.1 below).

The key assumption for a causal interpretation of our estimates is that the control group provides a realistic counterfactual of how much employment would have fallen in the absence of JobKeeper. This is the parallel trends assumption. We do a range of robustness checks on this assumption in our paper.

5.1 Worker Eligibility versus Actual Take-up

As discussed in Section 2.3, to receive JobKeeper a job had to satisfy two tests: (i) worker eligibility and (ii) firm eligibility. Failing either of these tests meant the job was ineligible for JobKeeper. It was possible for eligible firms to have both eligible and ineligible employees on their payroll.

In our data, we observe whether a person would have passed the worker-eligibility test in their main job. However, we do not observe whether they were also employed at an eligible firm because the LFS does not collect the necessary firm-level data. We also do not observe whether the person actually received JobKeeper. For this reason, as discussed above, we start by estimating the effect of JobKeeper worker eligibility on employment.

The effect of JobKeeper worker eligibility on employment will be an underestimate of the effect of JobKeeper on employment. This is because some fraction of those who were worker-eligible did not actually receive JobKeeper, either because they worked at an ineligible firm or because their employer did not enrol in the program.^[16] However, it is possible to scale up our estimates by the program ‘take-up rate’ (which we define in this paper as the share of worker-eligible employees who actually received JobKeeper) to obtain an estimate of the effect of JobKeeper on employment, which is the parameter of most interest to policymakers. Due to data limitations, our calculation of the take-up rate is not based on exactly the same group of workers as we study in the difference-indifferences analysis (i.e. casual employees with 6–23 months of job tenure in February). We discuss this further in Section 6.2. But first, we outline our approach to estimating the effect of JobKeeper worker eligibility on employment.

5.2 The Effect of JobKeeper Worker Eligibility on Employment

Our analysis focuses on a sample of employees who were employed on a casual basis immediately prior to the JobKeeper program. We focus on casual employees, rather than the broader population of employed people, because within the pool of casual workers it is possible to identify some workers who were potentially eligible for JobKeeper and other, otherwise similar, workers who were not.

Specifically, we compare casual employees who had 12–23 months of tenure in February 2020 and so were potentially eligible for JobKeeper (the treatment group) to casual employees who had 6–10 months of tenure in February and were therefore not eligible for JobKeeper (the control group). These groups should be similar on average, because they are all employed on a casual basis and fall within a fairly narrow range of tenure. Some descriptive statistics provide support to this; the two groups are similar in a range of observable ways, such as their age, industry and occupational skill level (Table 1).^[17]

A textbook difference-in-differences strategy does not require that the treatment and control groups be similar on average prior to JobKeeper, since any time-invariant group-level differences will be captured by the group fixed effect. However, having treatment and control groups that are balanced along a number of observable and unobservable dimensions prior to the program gives us more confidence that the assumption of parallel trends will hold. For example, if the two groups differed in terms of their industry composition prior to JobKeeper, we would worry that any differences in employment between the groups that emerged during the JobKeeper program period may simply reflect that the COVID-19 shock itself has had a very uneven impact across industries.^[18]

There were several considerations that went into choosing the specific tenure range for our analysis. The first was data. Our approach requires data on casual status and job tenure, which are collected in the February, May, August and November surveys. Tenure is measured in integer months for tenures from 0 to 11 months, and in integer years thereafter. Given that the February 2020 survey asked the respondent about their main job in the period from 2 to 15 February, people who reported having no more than 10 months of tenure in that survey would have had less than 12 months of tenure by 1 March (the key date for worker eligibility). This group can be unambiguously allocated to the control group, because they are ineligible based on the 12-month tenure rule. Those who reported having 11 months of tenure in the February survey may or may not have had 12 months of tenure by 1 March, and so we exclude this group from the analysis to avoid any ambiguity. Our treatment group are those employees with 12–23 months of tenure in February, which is the tightest window of tenure above the cut-off possible given the way the data is collected and coded.

The second consideration was the trade-off between bias and efficiency. Selecting a narrower range of tenure (say 9–10 months on the left-hand side of the 12-month cut-off) would mean that the treatment and control groups are more similar on average, and thus our main identifying assumption is more likely to hold. However, this comes at the cost of a smaller sample size. The tenure range that we chose for our baseline results balances these competing considerations.^[19]

The amount of tenure needed for a casual employee to be worker-eligible for JobKeeper coincides with a few other tenure-based thresholds in the Australian workplace relations system. After 12 months of employment at a firm, a casual employee can request flexible work arrangements or take unpaid parental leave (FWO 2020a). Some awards and agreements also allow a casual employee to request to become a permanent employee after 12 months of tenure at the firm, which can only be refused by the firm on ‘reasonable grounds’ (FWO 2020a).^[20] Our analysis assumes that these other options that open up to casual employees after 12 months of job tenure did not have a material bearing on a worker's employment outcomes during the COVID-19 shock.^[21]

An alternative to our difference-in-differences approach would be to use a regression discontinuity design (RDD). In principle, a researcher could use a RDD to exploit the discrete change in eligibility around the 12-month tenure cut-off. However, data constraints prevent us from doing this given that tenures of 12 months or longer are not collected in sufficiently granular intervals (e.g. months or days) in our dataset. Some administrative and proprietary datasets in Australia may be more fruitful for implementing a RDD, although access to these data is currently restricted.^[22]

The discontinuity in eligibility due to the 12-month tenure rule for casual employees is not the only source of variation in worker eligibility we can exploit to identify the causal effect of JobKeeper using the LLFS data. In Section 7.4 we also exploit differences in worker eligibility arising from the residency requirement. This alternative identification strategy provides a useful sense check on our main approach, but suffers from some additional data issues and biases that make it more difficult to interpret than our preferred approach based on the 12-month tenure rule for casual employees.

5.3 Estimation Sample

We exclude from our sample any people who worked in industries not eligible for JobKeeper. This includes the public sector and major banks.^[23] We are left with a sample size of 480 people in the treatment group and 367 in the control group by May. The sample size declines a little further in June and July, reflecting attrition from the sample which we assume occurs at random.^[24] Although our sample is not nearly as large as the administrative datasets available to some agencies, it is large enough to get reasonably precise estimates of our main parameters of interest.

Our plan is to extrapolate the findings from this sample of casual employees to the broader population of JobKeeper recipients. That is, we will assume that JobKeeper has a similar effect on employment for casual employees as it does for permanent staff. We discuss the reasonableness of this assumption in Section 8.3.

The focus of our paper is the first four months of the JobKeeper Payment program, as covered by the four monthly labour force surveys from April to July 2020. There are two reasons why we do not extend our analysis beyond this point. First, in early August the Australian Government announced changes to the scheme in response to the implementation of stage 4 social distancing restrictions in metropolitan Melbourne and stage three restrictions across regional Victoria (Morrison and Frydenberg 2020). Notably, the relevant date for assessing whether an employee was eligible for JobKeeper (in terms of being on the firm's books and having 12 months of tenure in the case of casuals) shifted from 1 March to 1 July.^[25] This decision, which broadened eligibility for the scheme, meant that some people we had classified to our control group were now eligible for JobKeeper, thus complicating our identification. Second, our sample size quickly diminishes to uncomfortable levels as we extend our analysis beyond July, reflecting the 8-month rotating panel design of the LFS.

5.4 Estimation Equation

We use the following model to estimate the effects of JobKeeper worker eligibility on employment:

where t = Feb 20, j; j = Nov 19, Dec 19, Jan 20, Mar 20, Apr 20, May 20, Jun 20, Jul 20 and i denotes individuals. E_i,t is a binary variable that equals one if person i is employed in month t, and zero otherwise.^[26] Elig_i is a binary variable that equals one if person i had 12–23 months tenure in February 2020 (potentially eligible for JobKeeper) and zero if they had 6–10 months tenure (ineligible for JobKeeper). c_i is an individual fixed effect and d_t is a time dummy that equals one in month j, and zero in February 2020.

We estimate the model separately for each month j, for all months spanning November 2019 to July 2020. We restrict our sample to individuals who were employed on a casual basis in February 2020 and also responded to the survey in month j. That is, for each month j we use a balanced panel for estimation, although the size of the panel differs for each period j due to attrition. Casual workers with tenure outside the 6–23 month range, or exactly 11 months, are dropped.

Our parameter of interest is $\delta$, which is the effect of JobKeeper worker eligibility on employment.^[27] Month-by-month estimation means $\delta$ can vary by month, thus tracing the dynamic effects of JobKeeper over time. We also estimate the ‘effects’ of JobKeeper in pre-treatment months as a robustness test.

Rather than estimating Equation (1) in levels, we estimate the model after taking differences (over worker i) from February 2020 to month j,

where $\sigma \equiv \gamma +1$ (making use of the fact that $E_{i,Feb20}=1,\forall i$) and $v_{i,j}\equiv {\epsilon }_{i,j}-{\epsilon }_{i,Feb20}$. In other words, our approach boils down to a regression of employment status in month j (e.g. July 2020) on whether the individual was worker-eligible for JobKeeper based on their casual status and job tenure in February 2020. We estimate Equation (2) month-by-month using a linear probability model with robust standard errors.^[28]

5.5 Measuring Employment

In the LFS, the ABS determines whether a person is ‘employed’ based on a framework that is in line with international best practice (ABS 2018). Examples of people who are classified as employed include:

• those who did at least one hour of paid work during the past week;
• those on paid leave while not working;
• those who are away from their job for less than four weeks, and believe they still have a job to go back to; or
• those away from their job for four weeks or longer but are paid for some part of the previous four weeks.

Under this framework, people paid through the JobKeeper wage subsidy will be classified as employed, regardless of the hours they work (e.g. even if they are stood down). While the survey does not collect information on whether the person was receiving JobKeeper, the ABS (2020d) expects that those who are paid through the JobKeeper scheme will answer the questions in a way that results in them being classified as employed, irrespective of whether they were stood down or at work.^[29]

Given this framework, some readers may think that the answer to our question of ‘what is the effect of receiving JobKeeper on employment?’ is trivial. That is, if everyone who receives JobKeeper will be classified by the ABS as employed, doesn't that mean that receiving JobKeeper has a one-for-one effect on employment? This is not the case. What we are interested in is the question of what would have happened to people who received JobKeeper in the counterfactual situation in which they had not received it. If some of those people who received JobKeeper would have remained employed regardless of the subsidy, then the effects will be less than one-for-one.

Our focus is employment, not jobs. A person who had multiple jobs prior to JobKeeper will remain employed if they held on to at least one of those jobs during the crisis (and met the criteria for employment in that job). The jobs versus employment distinction is important because some workers held more than one job, but each person could only receive JobKeeper from their primary employer. The extent to which JobKeeper cushioned the fall in jobs is not something we study in this paper.^[30]

Footnotes

If a person did not pass the worker-eligibility test in their main job, they may still have passed in a second job. We discuss this in Appendix D.6, along with the results of a robustness test that suggests this does not create a material bias. [15]

Some people who were JobKeeper worker-eligible in their main job may not have received the JobKeeper payment for that job if they instead received the payment for a job that was not their main job. But, in that case, the person still received JobKeeper. The individual's main job (as defined in the LFS) can differ from their primary employer (as defined by the ATO). [16]

Table C1 compares the descriptive statistics in Table 1 with equivalent descriptive statistics for the sample of all casual employees in February 2020 (including those with less than 6 months of tenure or more than 23 months of tenure) and the sample of all employees in February 2020 (including all casual and non-casual employees). [17]

An alternative approach would be to control for these pre-treatment differences directly (interacted with the time dummy). However, introducing control variables will only account for observable differences between the treatment and control groups, not the unobservable differences that may have affected employment outcomes in the absence of JobKeeper. [18]

In Appendix D.4, we present evidence that an employee's job tenure as at February 2020 was not predictive of his or her probability of remaining employed over the May to July period, for those people with 1 to 10 months tenure. This suggests that our baseline results would be robust to varying the width of the tenure window on the left-hand side of the 12-month cut-off. [19]

Some key awards that include this provision include the Hospitality Industry (General) Award [MA000009], Fast Food Industry Award [MA000003], General Retail Industry Award [MA000004], Hair and Beauty Industry Award [MA000005] and the Real Estate Industry Award [MA000106]. What constitutes ‘reasonable grounds’ to refuse the request varies by award, but can include: the employee does not work regular hours, the employee's job will not exist in the next 12 months, or the employee's working hours will be significantly reduced in the next 12 months. In other awards, a casual conversion request can be made after 6 months, such as the Manufacturing and Associated Industries and Occupations Award [MA000010] and the Building and Construction General On-site Award [MA000020]. [20]

The 12-month threshold for requesting flexible work arrangements, unpaid parental leave or permanent employment is not anchored at a fixed point in time (unlike the threshold for JobKeeper eligibility which was anchored at 1 March), which means that some of the lower-tenure group became eligible for these options during the April to July 2020 period. [21]

The key identifying assumptions of RDD are likely to be satisfied in that there was very limited scope for employers to manipulate the job tenure information given to authorities (see Section 2.3). Using these other datasets it may also be possible to exploit discontinuities in the revenue decline cut-offs in the firm-eligibility test. In saying that, the revenue decline cut-offs were fuzzier than the cut-offs that applied to individual workers, particularly given that firm eligibility could be based on projected, rather than actual, revenue losses, and that alternative tests could also apply. [22]

The LLFS micro data does not include a variable for whether the employee works in the public sector, so instead we drop employees in 3-digit industries where more than 60 per cent of employees are employed in the public sector (based on data from the 2016 Census). In addition to all industry sub-groups within the Public Administration & Safety industry division, this includes employees working in hospitals, tertiary education, rail and some utilities. The LLFS micro data does not identify whether the employee works for a major bank, so we drop any employees who works in the 3-digit Depository Financial Intermediation industry. Industry in the LLFS pertains to the individual's main job. [23]

By June, there were 377 people in the treatment group and 274 people in the control group. By July, there were 273 in the treatment group and 189 in the control group. More than two-thirds of people in February 2020 had left the panel by July. Much of this sample loss reflects the 8-month rotating panel design of the LFS, which reduces the initial sample by roughly one-eighth for every month we extend our analysis into the future. By July, sample rotation accounts for over 90 per cent of all exits from the survey panel relative to February, while premature attrition (attrition not explained by rotation) accounts for the remainder. While sample rotation leads to some data being ‘missing at random’, a potential source of bias in our study is the possibility that premature attrition does not occur at random. While our treatment and control groups are balanced across many of the observable dimensions that tend to be associated with attrition (e.g. age, occupational skill level), it is possible that premature attrition is also correlated with changes in a person's employment status, and if so, it can create a bias in our regression estimates. One source of premature attrition happens when households change their address, because the ABS drops those households from the survey. Because regional mobility is often associated with a change in employment status (e.g. a person who loses their job may relocate to a new region to find work), this non-random premature attrition could lead to a bias in our estimates. However, during COVID-19 this ‘mobility bias’ is likely to be smaller than normal, due to the constraints on moving home during the pandemic. In July, the premature attrition rates were higher for our control group (8.2 per cent of the February control group sample) than for our treatment group (5.7 per cent of the February treatment group sample). As such, if we assume those who prematurely left the panel did so as a result of job loss, our estimates of the effect of JobKeeper on employment would be understated. Indeed, if all the people who prematurely left the survey panel did so because of job loss, adjusting our difference-in-differences estimates to account for this (by setting the employment status of all premature leavers to ‘non-employed’ rather than ‘missing’) would yield estimates of the effect of JobKeeper worker eligibility on employment equal to 12.9 percentage points, which is higher than our baseline estimate of 8.2 percentage points (Figure 2). The bias will run in the opposite direction if premature attrition is negatively associated with employment. [24]

These changes were announced on 7 August 2020 and effective from 3 August. The August LFS referenced the period 1–15 August. [25]

A person who is not employed can either be unemployed or not in the labour force. We do not distinguish these states. [26]

We are being loose with our terminology here. Our difference-in-differences estimate actually yields the effect of JobKeeper tenure eligibility. This is slightly different to the effect of worker eligibility because the latter also requires the worker meet the residency requirement. [27]

Clustering the standard errors at the 2-digit ANZSIC 2006 industry level (at least 52 clusters over May to July 2020) produced standard errors that were 26 per cent (May), 18 per cent (June) and 6 per cent (July) smaller than robust standard errors that do not allow for intra-industry correlation. As such, our decision to use robust, rather than cluster-robust, standard errors is conservative. [28]

This is consistent with the long-standing concepts and practices used in the LFS. For more information, see ABS (2020d). [29]

The ABS payroll data suggest that the number of secondary jobs fell sharply relative to main jobs during our period of analysis (ABS 2020c). In Appendix D.6, we provide some discussion on the effects of multiple job holding on our results. [30]