RBA Annual Conference – 1998 Discussion

1. Bruce Chapman

John Martin has written an excellent overview paper on the nature, role and consequences of active labour market policies. He considers the (strong) rationale for active versus passive labour market expenditure, describes the main results from an extensive evaluation literature, and offers practical suggestions concerning successful program design. I am confident that the paper will become a widely cited reference for this important policy area; it is a major contribution to the conference.

The paper is accurate and balanced, which can be a dilemma for a discussant. This is resolved in what follows by offering some complementary observations and some suggestions for an alternative research approach. I do this with respect to the critical policy question: do labour market programs work?

Martin reports that there are two broad ways this issue is addressed in the literature. One is to consider the effects of program expenditure on aggregate labour market outcomes, such as a country's unemployment rate. The other is through microeconomic – usually econometric – studies of the effect of specific initiatives. What follows questions both, and suggests a new methodology for understanding the conceptual basis of programs and their implications for government expenditure.

Macroeconomic studies

The accepted evaluation methodology assumes that a major aim of active labour market expenditure is to increase the aggregate number of jobs. In a macroeconomic context – both within and between countries – much research focuses on the extent to which the overall unemployment rate falls with increases in program activity. A point hinted at by Martin, but not comprehensively explored, is that it is difficult to believe that such research can tell us much. It is worth drawing this out fully.

Macroeconomic approaches to evaluation are unconvincing, essentially because of the great difficulty of measuring the impact of relatively small and heterogeneous policies on a significant aggregate statistic. There are four main problems, now considered.

The potential size of the effects of labour market programs

A very useful contribution of Martin's paper is the documentation of active labour market expenditure in OECD countries. The data are presented both as proportions of GDP and relative to the number of unemployed people. The dimensions are instructive: in no country is active labour market expenditure per year as a proportion of GDP greater than 2.5 per cent, and the average is about 1 per cent. These data allow the first point to be illustrated: the potential effect of changes in program outlays is just not big enough to be identified in aggregate analyses.

Imagine that in response to a large increase in the unemployment rate in a particular year – say of 4 per cent – a government significantly expands active labour market program expenditure from, say, 0.75 to 1.25 per cent of GDP per annum. To illustrate how unlikely this is, consider the case of Australia. An increase in active labour market program outlays of 0.5 per cent of GDP means, currently, about $2.5 billion per annum. This is a big number.

Not even with the considerable re-orientation towards intervention that came with the implementation of Working Nation in 1994 did such a large increase occur. Specifically, the radical changes in active labour market program expenditure at that time were not more than about half of this.

Even if there is an increase in intervention of this order, and policies work to the maximum possible employment-creation potential, the orders of magnitude of their possible consequences are such that it would be quite hard to identify effects. In the context of understanding program influences for a phenomenon as complex as unemployment, it is not credible to believe that program implications could be measured accurately, if at all. There are obvious reasons for this imprecision, with several now considered.

The endogeneity of labour market policy and unemployment

Martin accurately identifies that active labour market expenditure is endogenous to the state of the macroeconomy, and that this is one of the most important difficulties for aggregate evaluation. The problem is that governments respond to actual or expected increases in unemployment by expanding active labour market policies. Similarly, as the unemployment rate falls, the perceived need for maintaining relatively high outlays on active labour market programs dissipates, and with this so too does expenditure fall.

There is thus an obvious cause and effect interpretation for the aggregate evaluation exercises, with an endogenous government response biasing results towards finding positive associations between the level of unemployment and active program expenditure. Even if the programs decrease unemployment, the policy endogeneity issue will make such a conclusion less likely to be found.

Compositional issues in measuring program effects

There are myriad types of labour market policy interventions, and Martin identifies five of these. A significant point for aggregate evaluation is that the effects of heterogeneous programs will be quite different in terms of a host of possible outcomes and costs. For example, some approaches are more likely than others to generate new jobs instantly, while others entail a participant acquiring employment at the expense of a non-targeted person. As well, the costs of different types of schemes differ markedly.

However, an implicit assumption in much of the aggregate evaluation research is that the employment ‘bang per buck’ is equivalent across programs. But so long as program composition changes over time, and/or differs between countries, this cannot be true. And the further is the assumption from the reality, the more unlikely it is that this type of research will find that aggregate measures of program expenditure are associated with decreases in unemployment.

The timing issue

A quintessential time-series econometric issue is that exogenous variables can have important effects, but with lags. Indeed, the timing of policy influence is mostly not obvious from economic theory and is thus usually difficult to determine. The problem is very clear with respect to the role of active labour market expenditure for the macroeconomy. A few examples are worth noting.

When a government institutes significant changes to its labour market policies, there might be an important lag between increases in outlays and the on-the-ground implementation of new or expanded programs. For example, setting up an administrative infrastructure and disseminating information will entail expenditure increases with no immediate returns.

As well, programs will have different timepaths for participants' employment prospects. For example, a public sector job scheme might improve the eventual job prospects of participants some time after the program ends, while a wage subsidy scheme could improve job prospects for participants immediately.

These distinctions are typically not made in the aggregate evaluation literature. This implies, again, that much progress needs to be made before any clear conclusions arise as to the effect of active labour market interventions on the macroeconomy. As Martin notes, we know very little in this area.

Microeconomic evaluation of programs

Given the extraordinary difficulties associated with the measurement and interpretation of the aggregate evaluation of programs, the question then turns to microeconomic assessment. This area of research is well documented in Martin's paper. Its essence is accepted generally as the right way to think about the issue, and is as follows.

The net employment-creation effects from programs are given by the equation:

Total new jobs = Gross program jobs − deadweight − displacement

where ‘gross program jobs’ is employment of participants, ‘deadweight’ is the number of targeted people who would have gained employment in the absence of the program, and ‘displacement’ refers to those who lost, or did not find, employment because a program-eligible person was employed instead.

Martin documents the general findings from research based on this methodology, and finds a very large variance in research results concerning program effects. His broad conclusions are that: generalisations about the effects of programs are not credible; active labour market programs are not a panacea for the aggregate unemployment problem of OECD countries; and, some types of programs have a useful role to play in increasing net employment.

Martin's conclusions are circumspect and based firmly on a plethora of professionally competent studies. However, such an apparently broadly based consensus should not necessarily be seen as the bottom line. A significant point now developed is that the research essence of the conventional approach is open to question; this is the most important contribution of this commentary.

The problem with the accepted methodology

There are good reasons to question the usefulness of the accepted methodology. Its important limitation is that it ignores the dynamics of displacement, deadweight and the effectiveness of programs. What happens over time is not accounted for, and might very well be critical to the assessments documented by Martin. This is an important point pertinent to all aspects of the effects of programs, and the dynamic aspects are now explained.

With respect to displacement, what matters is not that a proportion of those displaced become or remain unemployed. The issue is rather how long they remain unemployed and what their increased unemployment duration implies for eventual employment probabilities. That is, if the timeframe is not instantaneous, there must be different conclusions.

Similarly, with respect to deadweight, it is not very interesting to know that a program participant would have got a job without the program. What is more important is the probability that such a participant has their eventual employment prospects changed, and by how much, from being involved in the program.

Finally, the issue of program effectiveness is not well addressed with the conventional approach. Little rigorous account is typically taken of what happens after the program ends, in terms of changed employment probabilities. Again, having a longer timeframe of evaluation allows this issue to be taken into account.

In summary, the conventionally accepted evaluation method reported by Martin misses critical aspects of programs because it ignores effects beyond the initial period. The case for a more sophisticated approach is clear.

Improving the conventional evaluation method

A useful demonstration of the above problem, and its possible solution, has been developed by Piggott and Chapman (1995), and entails the use of a simple three-period flows model of unemployment. It provides a basis from which to account for the probability of programs changing the exit probabilities to employment over time of participants including the deadweight and those displaced. The considerable advantage of such an approach is that it explicitly takes into account what happens in the labour market after the initial period of program initiation.

Illustrations of what different scenarios of program effectiveness, deadweight and displacement mean in this dynamic scenario are provided in both Piggott and Chapman (1995) and Chapman (1998). They imply quite different assessments of the role of programs to those documented in Martin, although the broad thrust of his conclusions remain solid. As well, for the first time, the flows approach illustrates the public expenditure implications of program composition taking into account the additional tax revenue and lower welfare payments from net employment creation.

The bottom line with respect to method is that taking into account the dynamic aspects of programs leads to a significantly enriched understanding of what active labour market programs are all about. The current static evaluation processes miss a lot of the action, and it is time that they are improved.

It was not the job of John Martin to challenge orthodox method, and it should be recognised that the task assigned to him of reporting research findings has been done excellently. But if the flows methodology alluded to above, and reported fully in Piggott and Chapman, has an important story to tell, it is an obligation of the OECD to now explore its potential. There is a case for a different evaluation methodology which takes account of fundamental dynamic issues.

Summary

John Martin has provided this conference with a significant reference point concerning the role of active labour market policies in the OECD. With respect to how important programs are for net employment outcomes, the flavour of his paper is ambivalent but not jaundiced.

The important point to take from his overview is that the strength of the case for active labour market intervention is highly contingent. Successful outcomes depend on: the circumstances of the economy; the nature of the program; the effectiveness of implementation; and the existence and efficacy of supporting policies. There is no doubt that this is right.

The significant point of departure offered in this commentary concerns the methodology used in most of the research reported in Martin. As usually modelled, program effects are considered in a static context, as if the implications for those affected after the initial policy intervention are irrelevant, and this cannot be correct. An alternative conceptual framework is available.

References

Chapman, B. (1998), Would Increasing the Skills of the Unemployed Solve Australia's Unemployment Problem?, paper presented at the Centre for Economic Policy Research and Reshaping Australian Institutions Conference, ‘Fairly Efficient: Equity and Productivity in the Australian Labour Market’, Australian National University, Canberra, June.

Piggott, J. and B. Chapman (1995), ‘Costing the Job Compact’, Economic Record, 71(215), pp. 313–328.

2. General Discussion

Discussion in this session ranged across the following issues:

• the impact of labour market programs on different population groups;
• the evaluation of labour market programs;
• the role of profiling; and
• the privatisation of job placement.

Participants noted that labour market programs appear to be successful for some groups in the population but not others. It was agreed that the failure to understand why this is so constitutes a large hole in the profession's knowledge, which should be remedied. Some participants noted that labour market programs are particularly successful for sole parents, and this group has derived considerable benefit from their entry to employment and has subsequently moved up in the income distribution. It was suggested that the relative success of programs directed towards this group is, in part, a consequence of sole parents being relatively more labour-market ready, and in part, a result of the greater availability of jobs that such people were likely to seek.

On the other hand, labour market programs are particularly unsuccessful for young people. However, there is some tentative evidence that early intervention (that is, at the primary school stage) may be more successful than programs that focus on young people who have already lost contact with the formal education system.

Participants argued that it is necessary to examine labour market programs in combination rather than in isolation. Other participants argued that it is important to evaluate programs taking into account the macroeconomic environment in which the programs are operating. Furthermore, it is also important to consider the macroeconomic consequences of the programs, for example, whether they can permanently reduce the natural rate of unemployment by reducing (say) the incidence of skill bottlenecks.

It was also stressed that short-term evaluations may give a misleading impression of the effectiveness of programs. In the longer term it is important to take account of all the benefits that might be generated, including more indirect effects such as a reduction in crime. However, it is often difficult to assess the impact of labour market programs given that many of them are subject to frequent modification or even complete abandonment before they have been operating for long enough to have their full effect.

Others pointed out that longer-term evaluations in other countries suggest that workers gain from an initial boost in earnings as a result of participation in some labour market programs, but there is little evidence of an ongoing gain in earnings, although weekly hours of work appear to increase in the longer term, so that overall income of participants is higher.

Profiling of participants in labour market programs was highlighted by some as an important component of the implementation of labour market programs. Profiling involves identifying particular characteristics of eligible participants and tailoring the labour market programs to suit those characteristics. Others noted that the international evidence on the effectiveness of profiling is, however, mixed.

Finally, there was a discussion of the recent privatisation of job placement in Australia. It was noted that many countries are taking a great interest in the Australian experience. Participants agreed that it is important to ensure that the process did not result in ‘cherry picking’ – that is, job-placement agencies devoting resources only to those with high probabilities of finding employment. Therefore, it is essential to monitor the effectiveness of the incentive structure facing the job-placement agencies to guard against this. Some participants also suggested that while there are certainly benefits from increased competition among job-placement agencies, there is the potential for costs flowing from the loss of centralisation that existed under the previous system.