Active Labor Market Programs

Evidence from
Evaluations*
As many governments in developing and transition
countries grapple with growing unemployment and
underemployment, they are increasing turning to active
labor market programs (ALMPs) to reduce the risk of
unemployment and to increase workers' earnings capacity. Such programs work by enhancing labor supply
(through training); increasing labor demand (through
public works or subsidies); and improving the functioning of the labor market (through employment services).
ALMPs are often targeted to the long-term unemployed, workers in poor families, and particular groups
with labor market disadvantages. The programs not only
have economic objectives, but important social objectives, as well. OECD countries have a long and extensive
experience with ALMPs. However, the programs are
becoming more relevant in developing and transition
countries, too.
How much reliance should countries place on active
labor market programs? This is a controversial question.
Proponents argue that the programs are the most direct
instrument for dealing with unemployment and poverty among workers. Opponents counter that ALMPs are
largely a waste of public funds and that any observed
benefits for participants are usually at the expense of
other workers. Against this backdrop, it is important to
rigorously evaluate the impacts of these programs and
their cost-effectiveness,Governments should be realistic
about what ALMPs can achieve and allocate resources
on the basis of cost-effectiveness.

--

As this study shows, the evaluation picture is mixed.

Despite this, governments have little choice but to use
active programming as one instrument in their response
to unemployment and poverty in the labor force. The
challenge, then, is to learn from existing experiences,
investing in programs that have positive returns and
altering or dropping programs that do not. Hf ALMPs
are going to be an economically useful policy, it is very
important that governments carefully evaluate their
own programs and introduce interventions on the basis
of what works in their own country.

Evaluation Methodology: Issues and Key

Concepts
There are different types of program evaluations.
Process evaluations focus on how well a program is
delivered. Performance monitoring is concerned with
timely indicators of how well program objectives are
being achieved. These are very important tools for
improving program effectiveness and can be carried out
even in countries with low administrative and analytical
capacity. However, the incremental value of a program
and its cost-effectiveness can be calculated only through
impact evaluations. These impact evaluations, when
conducted rigorously, can identify the effects of a given
program on participants, and when coupled with cost
information, can reveal the net benefits of programs to
participants, to government finances, and in some cases
to the broader labor market and society. However, reliable impact evaluations require careful methodological

"This note was prepared by Nancy Morrison, based on Betcherman, Olivas, and Dar (2004). The World Bank Employment Policy
Primer aims t o provide a comprehensive, up-to-date resource on labor market policy issues. The series includes two products:
short notes, such as this one, with concise summaries o f best practice on various topics; and longer papers with new research
results or assessments of the literature and recent experience. Primer papers and notes are available on the labor markets Web
site a t ~www.worldbank.org/labormarkets,, by contacting the Social Protection Advisory Service at (202) 458-5267, or by email
a t ~socialprotec~on@worldbank.org~.

design, good data, and, for some methods, sophisticated

econometric techniques.
Generally, the central questions for ALMP impact
evaluations are:
What are the impacts of program participation
on the future labor market outcomes of participants?
661

What is the cost-effectiveness of programs?

Unfortunately, most evaluations focus on the first

question only; very few adequately address the cost
question. In most ALMP evaluations, the key impact
indicators are post-program employment rates and
earnings. However, some evaluations do look at other
social indicators such as criminal rates, teenage pregnancy, and receipt of government benefits.
Earlier reviews of impact evaluations by the World
Bank, the OECD, and others have concluded that policymakers must be cautious regarding what ALMPs can
realistically achieve. The evidence suggested that these

programs were not a panacea for unemployment but

that some types of interventions, properly designed,
could be effective for some workers. These reviews were
based almost exclusively on the experience of industrialized countries because very few evaluations existed anywhere else.
A more recent study (Betcherman, Olivas, and Dar
2004) takes a special look at developing and transition
countries. It pays particular attention to the quality of
available impact evaluations and includes only scientific
evaluations (that is, those utilizing a control group). It
builds on the 72 scientific evaluations considered in a
previous World Bank study (Dar and Tzannatos 1999)
by adding 87 new studies (see table 1). The new study
pays close attention to transition countries and, to a lesser extent, developing countries, and asks whether the
findings of impact evaluations in industrialized countries apply in these contexts as well.

I m p a c t EvaLeration

Pin&ngs
This updated review does not change the overall
findings from the 1999 study on the impacts of ALMPs
in any fundamental way. A wide range of results can still
be found. Some programs demonstrate positive labor
market effects for participants. Others show either no
impact or even negative effects. Obviously, program
design and the context in which the program operates
matters a great deal (see table 2).

It is increasingly difficult to isolate impacts of particular types of programs because of a trend to provide
service in an integrated fashion (that is, as part of a
package of programs). Nonetheless, this review leads to
the following general conclusions:

Employment services. These services include

counseling, placement assistance, job matching,
labor exchanges, and other related services. They
generally have positive impacts on participants'
employment and earnings after the program.
Costs are relatively low, so the cost-benefit ratio is
often favorable. However, employment servicesat least by themselves-are of limited use in situations where structural unemployment is high
and there is a lack of demand for labor. Moreover,
there are some questions about the coverage and
effectiveness of these services in developing countries, where many labor market transactions are
informal.
Training for the unemployed. Participants often
benefit from these programs in terms of higher
employment rates, but not in terms of higher
earnings. The few evaluations in developing
countries paint a less favorable picture. Programs
seem to work best with on-the-job training and
active employer involvement. Results are more
positive for women than men.
Retrainingfor workers i n mass layoffs. These programs most often have no positive impacts,
although there are exceptions. The few successful
cases typically include a comprehensive package
of employment services to accompany the
retraining. However, these are generally expensive.
Training for youth. These programs are almost
always unsuccessful in improving labor market
outcomes, at least in developed countries. It

makes much more sense to invest earlier in the

education system to reduce drop-outs and other
schooling problems. While few studies have been
done in developing countries, evaluations in
Latin America find positive impacts for programs
that integrate training with remedial education,
job search assistance, and social services.
RI

Wage/employment subsidies. Most often these do

not have a positive impact and have substantial
deadweight and substitution costs (see glossary).
Targeting and monitoring may help, but at the
cost of reducing take-up rates.
Public works. These can be an effective short-term
safety net, but public works do not improve participants' future labor market prospects.
Microenterprise development/self-employmentassistance. There is some evidence of positive impacts
for older and better-educated workers. However,
take-up is low.

What can be said about the impacts of ALMPs in

developing and transition countries in particular? Many
findings from industrialized countries seem to apply
broadly to transition countries. However, this is not
always true in the case of developing countries-on the
basis of what is still a small sample of studies. The much
larger informal labor markets and weaker capacity to
implement programs in developing countries may limit
what some programs can achieve in terms of creating
formal employment or increasing wages. The few evaluations in developing countries for employment services
and training programs for the unemployed are less positive than the (much larger) body of evidence in the
OECD and transition countries. On the other hand,
some youth training programs in developing countries
have much more positive impacts than in industrialized
countries. It may be that such programs in these lowincome labor markets have more potential because
abundant supplies of skilled workers are not available.
The sample of evaluations outside the OECD is still limited, especially in low-income developing countries.
Further studies will be needed to confirm these initial
observations.
The ingredients for successful interventions seem to
apply for all countries. Good design features include a
comprehensive package of services; programs that are
oriented to labor demand and linked to real workplaces;

I m p a c t EvaIanation

Retraining for
workers i n mass
layoffs

positive impacts on employment and

earnings.

economy is good. Impact Limited where there

is a Lack of demand for Labor.

Developing countries: Very little evidence.

Costs relatively Low, so cost-benefit ratio
usually positive.

I n developing countries, questions about

coverage and effectiveness of these services,
given informal labor markets.

Often no positive impact on employment and

earnings, but there are exceptions.

Better results may be achieved with

integrated training and employment services.

Transition and developing countries: Very little

evidence.

Wage/ employment
subsidies

Developed and transition countries: Overall

negative impacts on employment and
earnings.
Developing countries: Very little evidence.

Recent evaluations i n developed countries

more favorable (such as welfare-to-work
programs).
Programs may be more effective when
combined with training.
Deadweight and substitution effects likely,
important but often not fully assessed.

Microenterprise/
self-employment
assistance

Not enough labor market-oriented

evaluations t o determine overall employment
and earnings impacts.

Very Low take-up.

some
of positive
impacts for
better-educated individuals.
Results likely better when technical and
advisory services accompany financial aid.

Impact Evaluation
and careful targeting. Finally, program impacts are usually more positive when the economy is growing.
While knowledge on the impacts of ALMPs continues to grow, there is still much more to learn, especially
in the context of developing and transition countries.
Evaluations rarely track post-program outcomes beyond
a couple of years, so little evidence exists on longer-term
impacts. Many studies do not estimate the deadweight,
substitution, and displacement effects and thus cannot
account for the general equilibrium impacts of programs. Many do not fully consider program costs and
thus cannot offer insights on the key policy issue of eEciency. Finally, the evaluation literature provides insights
into what works, but far less on why. Ultimately, policymakers need to understand what circumstances and
design features explain effective outcomes for specific
groups (see table 3).

Trends in A c ~ v eLabor Market Programs

in Indust~alizedCount~es
The strategy, design, and implementation of active
labor market policy in industrialized countries have
undergone some significant changes over the past

Reduce structural imbalances

Enhance skills and productivity

decade. Specifics differ by country but in most industrialized countries, many of the following trends are evident:
Increased emphasis on job search assistance. This
assistance often has been linked with closer monitoring
and tighter job search requirements for unemployment
benefit recipients.
Integrated services (one-stop window/guichet
unique) so that clients can receive information, counseling, and access to services from a single source.
An increasing reliance on private delivery of services: for example, for training, employment services, and
public works. Government's role in these areas has been
to establish overall priorities, ensure quality, and provide
financing, especially to address equity concerns.
Closer partnership with employers and communities to ensure that programs respond to market conditions.
"Making work pay" initiatives. These have used
employment subsidies and tax benefits primarily to
encourage low-skill workers to take low-pay jobs.

Training (subsidies or grants t o workers

or employers)
Self-employment support
Employment services (information,
search assistance, mobility assistance)
Training
Wage subsidies

Hard-hit regions and industries

Proximate regions, industries, or

occupations

Employment services
Training (apprenticeship, school-towork transition)
Training and retraining (including inservice, apprenticeship)

Training (grants, subsidies)

Wage subsidies

A t risk or disadvantaged worker

categories (especially for retraining)

Impact EvaLuation

"Profiling" of unemployed clients to identify those

likely to need employment services or retraining. In
some countries, program participation (based on profiling) is an obligatory condition to qualify for unemployment benefits.
Increased emphasis on program evaluation and
allocation of resources on the basis of evaluation results.
Developing and transition countries considering
these directions should carefully consider how well these
approaches could be implemented and would work in
their own countries.

References
Betcherman, Gordon, Karina Olivas, and Amit Dar. 2004.
"Impacts of Active Labor Market Programs: New Evidence
from Evaluations with Particular Attention to Developing
and Transition Countries." SP discussion paper No. 0402
January 2004, World Bank, Washington, DC.
Betcherman, Gordon, Amit Dar, Amy N. Luinstra, and Makoto Ogawa. 2000. Xctive Labor Market Programs: Policy
Issues for East Asia." Social Protection Discussion Paper
No. 0005, World Bank, Washington, DC.
Dar, Amit and P. Zafiris Tzannatos. 1999. 'hctive Labor Market Programs: A Review of the Evidence from Evaluations.'' Social Protection Discussion Paper No. 9901,
World Bank, Washington, DC.
World Bank. 2002. "Impact Evaluation: Techniques for Evaluating Active Labor Market Programs." Employment Policy Primer Series, No. 2. Social Protection Unit, World
Bank, Washington, DC.

Employment Policy Notes are published to communicate the results of The World Bank's work to the development community with the least possible delay. The typescript manuscript of this note therefore has not been prepared in accordance with the
procedures appropriate to formally edited texts. The findings, interpretations, and conclusions expressed herein are those of the
author(s), and do not necessarily reflect the views of the International Bank for Reconstruction and Development / The World
Bank and its affiliated organizations, or those of the Executive Directors of The World Bank or the governments they represent.
The World Bank does not guarantee the accuracy of the data included in this work. For free copies of this note, please contact
the Social Protection Advisory Service, The World Bank, 1818 H Street, N.W., Room G7-703, Washington, D.C. 20433-0001. Telephone: (202) 458-5267, Fax: (202) 614-0471, E-mail: socialprotection@worldbank.org or visit the Social Protection website at
www.worldbank.org/sp.