Final Synthesis Report On Access To Employment 12 Oct 2012

Final Synthesis Report on Access to
Employment
ESF Expert Evaluation Network

Training and Employment Research Unit (TERU)
University of Glasgow, Adam Smith Building
UK-Glasgow G12 8RT
and
Metis GmbH
Donau-City-Strae 6
A-1220 Vienna
October 2012
The views expressed in this report are those of the authors and do not
necessarily reflect the official opinion of the European Commission. All errors
or omissions remain the responsibility of the authors.
Final Synthesis Report on Access to Employment
ESF Expert Evaluation Network

Final A2E Synthesis Report
Contents
Contents ......................................................................................................................... 3
Tables and Figures ....................................................................................................... 5
1 Executive Summary .............................................................................................. 6

1.1 Background and study objectives ........................................................................... 6
1.2 Overall progress in delivery of ESF Access to Employment .................................. 7
1.3 Analysis of interventions ......................................................................................... 7
1.4 Factors influencing and good practice in implementation ....................................... 8
1.5 Community Added Value ........................................................................................ 9
1.6 Evaluation data, approaches and methodologies ................................................... 9
1.7 Conclusions........................................................................................................... 10
2 Background and study objectives .................................................................... 12

2.1 Background to Report ........................................................................................... 12
2.2 ESF and Enhancing Access to Employment (A2E) .............................................. 12
2.3 Crisis in economy and labour market ................................................................... 14
2.4 This Synthesis Report ........................................................................................... 20
3 Overall progress in delivery of ESF Access to Employment ......................... 23
4 Analysis of interventions and target groups ................................................... 29

4.1 General analysis ................................................................................................... 29
4.2 A2E interventions and target groups .................................................................... 29
4.3 Significant results achieved to date ...................................................................... 38
4.4 Effectiveness of ESF A2E interventions ............................................................... 44
5 Factors influencing and good practice in implementation ............................. 46

5.1 Factors inhibiting effective implementation ........................................................... 46
5.2 Factors contributing to effective implementation .................................................. 49
5.3 Good Practice in implementation .......................................................................... 51
6 Community Added Value ................................................................................... 56

6.1 Variation in CAV by priority group and type of intervention .................................. 61
7 Evaluation data, approaches and methodologies ........................................... 62

7.1 Need for good quality evaluation evidence ........................................................... 62
7.2 Evaluation data sources, quality and reliability ..................................................... 62
7.3 Evaluation approaches and methodology ............................................................. 67
page 3
7.4 A2E evaluation requirements and deliverability .................................................... 71
7.5 Some implications ................................................................................................. 73
Annex to Section 7: Analysis of evaluations based on Inventory ......................... 74
8 Conclusions......................................................................................................... 79
8.1 Positive messages ................................................................................................ 79
8.2 Challenges for ESF A2E ....................................................................................... 79
Appendix: Detailed Tables ......................................................................................... 81
Glossary ..................................................................................................................... 106
page 4
Tables and Figures
Table 1. Number and Volume of ESF OP by Member State ........................................ 23

Table 2. Financial and Physical Progress for ESF A2E, to December 2010 ................ 26
Table 3. Types of intervention ....................................................................................... 33
Table 4. Target Groups ................................................................................................. 36
Table 5. Overall Effectiveness of ESF A2E .................................................................. 39
Table 6. Effectiveness of A2E Interventions by Type and Target Group Selected
Member States ...................................................................................................... 41
Table 7. Evaluation data sources .................................................................................. 65
Table 8. Evaluation approaches and methodologies .................................................... 69
Figure 1. % Change in Employment Levels (Aged 15-64), 2007 to 2011..................... 18

Figure 2. % Change in Part-Time Employment (Aged 15-64), 2007 to 2011 ............... 19
Figure 3. % Change in Unemployment Levels (Aged 15-64), 2007 to 2011 ................ 20
page 5
1 Executive Summary
1.1 Background and study objectives

This is the second Synthesis Report from the ESF Expert Evaluation Network (ESF
EEN) established in September 2010. The network brings together experts in all 27
Member States to collect, analyse, aggregate and synthesise evaluations and other
relevant studies on the nature and contribution of ESF in each Member State. The
experts compile an inventory of evaluation plans, activities, studies and key findings as
well as reports on specific themes.
The first Synthesis Report produced an analysis of evaluation plans and activities
across the 27 Member States and identified the common features of the ESF
evaluation landscape. Enhancing A2E has been chosen as the first theme for the
thematic country reports, as this generated the highest number of findings in the first
synthesis report.
This second Synthesis Report assesses ESF-supported A2E across the 27 Member
States on the basis of country reports. Although the great majority of these are
thematic, some are contextual due to the lack of evaluation or related reports.
Access to Employment (A2E) is one of the key policy fields for the 2007-2013 ESF
Operational Programmes (OPs) and is described as: enhancing access to
employment and the sustainable inclusion in the labour market of job seekers and
inactive people, preventing unemployment, in particular long term and youth
unemployment, encouraging active ageing and longer working lives and increasing
participation in the labour market.
The regulation notes that ESF will support a number of actions to promote A2E.
Modernising and strengthening labour market institutions.
Implementing active and preventive measures.
Mainstreaming and specific action to improve A2E for women.
Specific action to increase participation of migrants in employment.
A key issue is that the 2007-2013 ESF OPs were designed at the end of a long period
of consistently rising employment levels and increasing tightness in labour markets
across Europe. In other words, the ESF A2E actions which were largely designed to
work on the supply side of the labour market were seen to be playing into economies
with labour demand characteristics that would be strongly supportive. This is a
particularly important consideration for members of priority groups (e.g. disabled
people, ex-offenders, and discriminated against minorities) whose relative labour
market experiences tend to improve when employment opportunities are more
buoyant.
However, the economic crisis which began in the summer of 2008 has created a
much more challenging environment for the effective implementation of ESF A2E. The
constraints on public sector budgets across Europe which are the second major
consequence of the economic crisis also pose challenges for ESF more generally by
making it more difficult to secure the necessary match funding. Finally, a number of EU
economies as a result of budget constraint have been re-designing both their active
labour market policy interventions and their passive social security systems with a view
to providing a lower cost service. As ESF A2E actions are often implemented through
co-financing national government mainstream programmes these constraints on
budgets are likely to impact on the effectiveness of ESF A2E key areas of intervention.
page 6
Although labour market challenges have been a pan-EU experience, the evidence is
that some Member States have enjoyed an increase in employment levels since 2007,
most have experienced a decline and a smaller number are trying to come to terms
with a substantial loss of employment. The implication of this is that the
appropriateness of ESF A2E actions will be even more variable across Member States,
and likewise with the effectiveness of A2E key areas of intervention in terms of
generating employment results.
1.2 Overall progress in delivery of ESF Access to Employment

Largely based on information generated in the AIR 2010 documents, progress on the
delivery of ESF A2E by December 2010 has been assessed.
Around 23 billion of public sector (ESF plus Member State) cost was committed
to ESF A2E key areas of intervention.
Over 12.5 million final recipients were supported by these interventions.
Around 2.4 million final recipients achieved an employment result, defined in
this report as going into a job after their ESF intervention. This estimate
combines figures for final recipients entering jobs after their ESF intervention, or
in a job typically 3 or 6 months after completing their intervention.
In only one country report was evidence noted on the sustainability of
employment enjoyed by final recipients who gained a job in the first place.
There is clear evidence overall that a substantial amount has been delivered through
ESF supported A2E with significant numbers of final recipients.
There are some issues in relation to progress.
There is too much missing information at this quite advanced stage of the
implementation of the 2007-2013 programming period, particularly on the simple
measure of the numbers of final recipients entering a job after completing their
intervention.
Evidence on whether employment has been sustained is available only on a
very limited basis.
There is substantial variation across Member States in relation to progress
against targets for final recipients engaged and numbers entering employment,
although this may reflect on the target setting process across Member States.
There is potential to explain variation in progress against targets by looking at
different architectures across Member States around the number of OPs and the
nature of the national and regional structures, but better recorded measures of
progress would be needed to make this analysis worthwhile.
1.3 Analysis of interventions

On the basis of the more focussed evaluations available on A2E key areas of
intervention and priority axes a number of conclusions emerged from the country
reports.
From around 2009, additional resource has been allocated or re-allocated to key
areas of intervention such as increased support for self employment and
enterprise starts, as well as to stimulate employment demand in private sector
page 7
enterprises and/or to develop intermediate labour markets, social firms and
related mechanisms.
Interventions to promote A2E fell into basically three categories: enhancing
recipient employability, supporting the creation and retention of employment and
building the capacity of public employment and training service organisations.
The prioritisation of ESF-funded A2E interventions focused principally on
disabled people, various groups of unemployed or inactive people (including
those with multiple disadvantages and complex needs), ethnic minorities,
migrants and asylum seekers, women and young people.
In terms of significant results achieved to date.
In most country reports where data were available results for employment entry
and sustainability were at or close to target. Although some experts argued the
targets were not particularly stretching, this needs to be balanced by a
consideration of the serious deterioration in the labour market from the point in
time at which the targets were initially set.
The results for employment entry and sustainability achieved varied significantly
across Member States. The difficulty with this analysis is the lack of sufficient
cohort data which take a group of recipients who have completed their
intervention and been in the labour market for sufficient time to measure
sustainable employment entry.
Employment results within Member States vary significantly across types of
intervention and priority groups.
1.4 Factors influencing and good practice in implementation

A number of factors were identified in the country reports as constraints on effective
implementation.
The effects of the economic crisis were cited in most country reports as
impacting significantly on the efforts to move people into jobs and to sustain the
employment results.
Most country reports also evidenced challenges for Managing Authorities (MAs),
Intermediate Bodies (IBs) and beneficiary organisations flowing from
management and administrative issues related to ESF.
In some instances, problems with the design of key areas of intervention were
highlighted as a constraint on effective implementation.
On a more positive note, a number of factors facilitating successful implementation
emerged from the country reports.
It proved possible to re-focus some OPs to take account of the changed labour
market circumstances flowing from the economic recession.
Good quality staffing, systems and management arrangements supported
effective implementation.
Where national employment and skills agencies had strong capacity in terms of
helping place final recipients into appropriate jobs, it proved easier to implement
ESF-funded A2E activities successfully.
page 8
Good practice in implementation was highlighted across a wide range of country

reports and this tended to revolve around the following delivery issues.
Engaging hard to reach priority groups.
Customising A2E activities to meet the needs of individual clients.
Careful designing of interventions around the needs and capabilities of specific
priority groups.
Linking individual needs and labour market opportunities in a way which
recognises that the individual and the employer are both valued customers.
Achieving added value from A2E activities by creating effective partnerships
of agencies and beneficiary organisations.
1.5 Community Added Value

Community Added Value (CAV) measures the difference made by ESF A2E funding
and programmes compared to the efforts of Member States alone. There are four key
dimensions: volume effects, scope effects, role effects and process effects.
The treatment of CAV in the country reports proved to be extremely patchy, and in only
two cases were all four aspects of CAV covered. Looking at each of the aspects:
Volume effects emerge as the most common form of CAV, particularly
significant for newer members of the EU. Volume effects have been heightened
by the recession which has increased the demand for A2E activities at the same
time as Member State budgets have been in decline in most instances.
Scope effects are mentioned in many of the country reports in connection with
support for specific groups (for example, immigrants) which would otherwise
receive limited or indeed no A2E activity as well as support for specific types of
intervention, particularly the more labour intensive and costly ones
Role effects focus on innovation and were much less frequently cited in the
country reports.
Process effects, in terms of impacts on the way institutions and systems are
organised, are again less frequently mentioned but are more important forms of
CAV in the newer Member States.
1.6 Evaluation data, approaches and methodologies

Experts reviewed the data sources used for the evaluation process.
Output data were most commonly used in the evaluations and other documents
reviewed.
Other forms of secondary data (monitoring, financial and final recipient
contact data) were also commonly used.
Results based on research were much less common.
In the main the experts conclude that:
Where data were available, they were relatively accessible to experts.
There were no major concerns with the reliability of the data available, although
changes over the life of the 2007-2013 programming period meant that data
were not always comparable over time.
page 9
In relation to results based on research, the experts flagged up variations in
sampling methods used, challenges in securing sufficiently large sample sizes
and attrition in cohort studies as potential issues.
However, it is clear from many of the country reports that experts struggled to find
data that related specifically to ESF A2E types of operations focussed on the types
of final recipients prioritised under A2E. This applied to expenditure, number of final
recipients and to employment results. On employment results, there was virtually no
information on the sustainability of employment secured following an ESF A2E
operation. Additionally, for employment results many country experts were dependent
on monitoring data provided by beneficiary organisations raising issues about the
reliability of the information.
In terms of the evaluation approaches and methodologies used in preparing the
country reports:
The most common approach was to build on monitoring data and evaluations
of management and implementation processes.
Evaluations drawing on primary research were much less common.
Impact evaluations were rare and where they did exist were more likely to be
theory-based than counter-factual or other impact evaluations. The limited
number of impact evaluations reflects the difficulties involved in designing and
delivering these, including accessing data particularly micro data; identifying,
recruiting and sustaining control groups including data protection issues; and
the stage in the programming period with most Member States focussing on
monitoring implementation rather than evaluating results.
Many evaluations did not provide even the most basic information on the results
(achieving sustainable job entry) and the cost of achieving these results.
1.7 Conclusions
Positive Messages
1. The key positive message is that by December 2010 a significant ESF A2E effort
had been mounted and delivered across the 27 Member States.
2. There is a lot of evidence across the country reports that, in relation to A2E, there
have been important changes in an effort to respond to the economic crisis. It is
important not to underestimate the challenge to MAs in trying to implement OPs
designed for the very different labour market circumstances which prevailed pre-credit
crunch.
3. There is a widespread appreciation that, particularly for clients with multiple
disadvantages (or multiple and complex needs), ESF A2E interventions should reflect:
the need for interventions over a relatively long time period before sustainable
employment results can be achieved; the importance of having access to a range of
services; the need for a much more customised or individualised approach and the
value of having a key worker/personal adviser/work coach/etc who works closely
with the individual over the course of their journey towards and into employment.
4. There is a growing understanding of the value of interventions which support
individuals once they have entered employment, particularly important in the
current economic crisis where labour market fragility is the norm.
page 10
Challenges for ESF A2E

5. In some instances, experts have questioned the added value of A2E ESF,
particularly in terms of co-financing Member State mainstream welfare to work
programmes where many of the recipients are relatively short-term unemployed. The
labour market recession raises issues about the most appropriate allocation of ESF
resources between those further from and closer to the labour market.
6. There is very little evidence coming through the country reports of a serious
appreciation at governmental level within Member States of the need to see better
service integration across public employment services, health services, social
services and other services which support A2E.
7. The treatment of CAV in many of the evaluations underpinning the country reports
appears to be superficial. Somehow or other more robust information on the CAV
associated with ESF A2E must be collated.
8. More generally, on the evidence of the country reports, the evaluation effort around
ESF A2E is fragmented and generally lacking in robustness. By this stage of the
2007-2013 programming period the Commission should be able to inspect much more
comprehensive and robust information on the results of ESF A2E across the 27
Member States. This finding needs to be seriously considered in the design of the new
ESF OPs post-2013.
9. In order to create an effective base for evaluations which measure effectively the
impact and CAV of ESF A2E, there is a requirement for more evaluations which:
Independently measure job entry and job sustainability results.
Benchmark job entry and job sustainability results.
These types of evaluation would help to answer the basic question is ESF A2E
getting significant numbers of people into jobs and helping them sustain these jobs?
page 11
2 Background and study objectives
2.1 Background to Report

This is the second Synthesis Report from the ESF Expert Evaluation Network (ESF
EEN) established in September 2010. The network brings together experts in all 27
Member States to collect, analyse, aggregate and synthesise evaluations and other
relevant studies on the nature and contribution of ESF in each Member State. The
experts compile an inventory of evaluation plans, activities, studies and key findings as
well as reports on specific themes.
The first Synthesis Report1 produced an analysis of evaluation plans and activities
across the 27 Member States and identified the common features of the ESF
evaluation landscape. Enhancing A2E has been chosen as the first theme for the
thematic country reports, as this generated the highest number of findings in the first
synthesis report.
This second synthesis report builds upon 27 country reports. The preparation of the
country reports is underpinned by the inventory of evaluation materials prepared and
updated by the country experts. The reports were based on inventories updated to
mid-June 2012. Whereas the great majority of the country reports are thematic which
assess ESF A2E on the basis of evaluation and other relevant and informative reports,
some experts were forced to prepare contextual reports due to lack of evaluation
material covering ESF A2E priority axes and operations.
2.2 ESF and Enhancing Access to Employment (A2E)

The ESF in the programming period 2007-2013 is set out in Article 3 of the ESF
Regulation No 1081/2006 and comprises the following policy fields:
Increasing adaptability of workers, enterprises and entrepreneurs with a view to
improving the anticipation and positive management of economic change.
Enhancing access to employment and the sustainable inclusion in the labour
market of job seekers and inactive people, preventing unemployment, in
particular long-term and youth unemployment, encouraging active ageing and
longer working lives, and increasing participation in the labour market.
Reinforcing the social inclusion of disadvantaged people with a view to their
sustainable integration in employment and combating all forms of discrimination
in the labour market.
Promoting partnerships, pacts and initiatives through networking of relevant
stakeholders, such as the social partners and non-governmental organisations,
at the transnational, national, regional and local levels in order to mobilise for
reforms in the field of employment and labour market inclusiveness.
Expanding and improving human capital.
Strengthening institutional capacity and the efficiency of public administrations
and public services at national, regional and local level and, where relevant, of
the social partners and non-governmental organisations, with a view to reforms,
better regulation and good governance specially in the economic, employment,
education, social, environmental and judicial fields.
The Regulation notes that ESF supports a number of actions to enhance A2E by
promoting:
1 Metis and Eurval (2011) ESF Expert Evaluation Network Synthesis Report 1
page 12
The modernisation and strengthening of labour market institutions, in particular

employment services and other relevant initiatives in the context of the
strategies of the European Union and the Member States for full employment;
The implementation of active and preventive measures ensuring the early
identification of needs with individual action plans and personalised support,
such as tailored training, job search, outplacement and mobility, self-
employment and business creation, including cooperative enterprises, incentives
to encourage participation in the labour market, flexible measures to keep older
workers in employment longer, and measures to reconcile work and private life,
such as facilitating access to childcare and care for dependent persons;
Mainstreaming and specific action to improve access to employment, increase
the sustainable participation and progress of women in employment and reduce
gender-based segregation in the labour market, including by addressing the root
causes, direct and indirect, of gender pay gaps;
Specific action to increase the participation of migrants in employment and
thereby strengthen their social integration and to facilitate geographic and
occupational mobility of workers and integration of cross border labour markets,
including through guidance, language training and validation of competences
and acquired skills.
With strong jobs growth for most of the 1990s and 2000s but continuing relatively high
rates of unemployment in particular regions and sub-regions and for specific groups of
the population such as the young and older unemployed, people with health issues and
disabilities, those with low or no qualifications and other groups excluded due to
discriminatory factors (e.g. ethnic minority groups), there was an increasing emphasis
on the need to design interventions that will improve A2E. Enhancing A2E in 2007-13
programming period accounts for approximately 30% of the ESF.
Access to good jobs is not equally distributed between all social groups and
individuals. For a variety of reasons relating to origin, education, training, social and
economic background, physical and mental health and age, some people find it more
difficult to get a job than others. For example, older workers, young people entering the
labour market, women returners, migrants and people with disabilities are all
particularly vulnerable in the labour market and increasingly so in times of economic
downturn. Youth unemployment of up to 50% plagues a number of Europes more
fragile economies and women still suffer discrimination in the labour market in terms of
job quality and equal pay. In this context it is apparent that A2E and social inclusion
are interlinked fields and the country experts will have to explain such overlaps in their
reports if they find it difficult to distinguish between the findings.
From extensive analysis of the factors which restrict access to employment for more
disadvantaged and vulnerable groups and also from the many evaluations of
interventions designed to promote their employability, it is now generally acknowledged
that enhancing access to employment is a complex process which requires
sophisticated and well targeted interventions.
A key consideration is that the groups of the population experiencing high levels
of unemployment due to limited access to employment will nonetheless require
different types of interventions depending on the obstacles they confront in
finding and sustaining a job.
page 13
For some groups, the principal requirement will be to provide them with the
vocational and other job related skills which meet the needs of current and
prospective employers in a rapidly changing labour market.
The most disadvantaged groups of the population seeking to enter or re-enter
the labour market typically require a spectrum of services and interventions
dealing both with personal and social issues and with vocational skill
requirements.
The implication of these issues is that ESF key areas of intervention designed for and
targeted at specific vulnerable and priority groups need to be based on a deep
understanding of the factors that restrict access to employment for the types of people
the intervention is designed to assist. One size fits all is not the effective way forward.
A2E also requires a range of different interventions depending on the needs of specific
groups, and even individuals within groups.
Those with relatively good employment characteristics such as high
qualifications may require a simple intervention such as a graduate trainee or
internship programme.
Individuals who have been unemployed for a long time may confront a series of
disadvantages and have multiple and complex needs. They may require a
significant number of different interventions over a long period of time.
Effective interventions for promoting access to employment need also to be delivered
across a range of labour market participants, in addition to individual participants who
are members of priority groups such as youth unemployed.
2.3 Crisis in economy and labour market

The strong growth in global employment across the EU from the mid 1990s through to
2007 provided a supportive socio-economic environment for closing the gap between
more disadvantaged groups and residents of more disadvantaged regions and sub-
regions and the average economic and employment experiences within Member State
economies.
Although there are variations in this trend across economies, the evidence suggests
that more disadvantaged groups and groups discriminated against in the labour market
made significant ground over this period of time.
The gap between male and female employment rates narrowed considerably,
particularly among the 25-54 age group.2
The employment rate amongst older people (aged 55 plus) also increased more
than the average working age employment rate.3
Nonetheless, sizeable gaps in key labour market indicators such as the risk of
unemployment and earnings remained. For some groups for example, those with low
or no qualifications there is less evidence that the position improved in tightening
labour markets. For other groups, the gap narrowing began to reverse before the onset
of the recession.
2 Ramb, F. (2008). Employment Gender Gap in the EU is Narrowing, Eurostat Statistics in Focus 99/2008
3 Eurostat (2012). Active Ageing in the EU, Eurostat News Release 8/2012
page 14
Employment levels of people with disability improved up to 2000 but then

declined overall, there was no improvement in the relative position between
the mid-1990s and mid-2000s.4
The gap in employment rates between low qualified people and the rest of the
working age population narrowed in around half of the member states but
overall the gap across the EU as a whole remained the same.5
This broad context of improving experiences in relation to group and spatial labour
market disadvantages provided significant scope for the 2007-2013 ESF OPs to work
in a focused and targeted way to reinforce the positive forces associated with
strengthening labour markets to accelerate the closing of the gap between the more
and less advantaged in the labour market. Within this, A2E activity was to be critical in
linking the growing opportunities in the labour markets with the more complex needs of
priority groups such as the long term unemployed
A critical change in the socio-economic context, however, was the onset of a serious
global recession from around the summer of 2008. At this point the most significant
changes were:
A rise in unemployment and in economic inactivity across all national
economies.
An associated reduction in the recruitment by employers making it most difficult
for those already out of the labour market trying to get in, i.e. young people,
people with caring responsibilities trying to re-enter employment, ex-offenders,
etc.
In effect, ESF-funded interventions were now having to prove their effectiveness in an
environment which switched radically from one where employers were keen to recruit
labour and were suffering labour shortages, to one where recruitment was at a much
lower level and employers had a greater choice of candidates.
Arguably the importance of enhancing A2E for priority groups has been heightened
with the onset of the global recession in 2008. The point here is that when there are
fewer jobs available competition becomes much fiercer for those jobs and so we need
to work much harder to enhance the employability of vulnerable and priority groups in
the labour market. On the other hand, as the problem of employment becomes
increasingly one associated with a substantial jobs deficit, as well as changes in the
structure in employment (e.g. with more part-time jobs) traditional A2E key areas of
intervention will inevitably struggle to achieve employment results. There is a difficult
balance to be struck given these competing tendencies.
The 2007-2013 OPs were designed before the onset of the recession, and this means
that OP implementation strategies were based on the needs and opportunities
associated with the pre-credit crunch labour market. The challenge in terms of making
the best use of later tranches of funding during the 2007-2013 programming period for
many economies has been how to maintain a targeted and focused approach on
groups and areas most in need of ESF resources to raise their competitiveness in the
labour market at a time when labour market opportunities were declining significantly.
In broad terms, the response has been to resource interventions that support the
economic recovery, for example by creating stronger focus on the employability and
vocational skills required in sectors of the economy projected to grow across the
4 OECD (2010). Sickness and Disability in Work: Breaking the Barriers
5 Eurofound (2009). Low Qualified Workers in Europe
page 15
European Union. A good example of this has been the employability pathway or
pipeline approach utilised by a number of Member States which has the dual merits
of:
Focussing resources on helping to prepare people for real jobs with good
sustainability in the longer term, and
Creating an employability pathway which provides support of various kinds for
more socially excluded groups of the population (e.g. health support, child care
assistance, etc) allied with the development of appropriate vocational skills for
growing sectors.
This pipeline approach can help facilitate the most effective deployment of ESF
resources alongside mainstream national government resources in the fields of health,
education and training. As yet, there are no systematic evaluations of the value added
by the employability pathway approach in practice.
A second major consequence of the global recession and the factors giving rise to it is
the retrenchment in public expenditure in a large number of Member States. As is
well known, this is particularly severe in economies such as Greece, Spain and
Portugal but there will be significant reductions over the remainder of the 2007-2013
ESF programming period. This means that the mainstream funding which is deployed
alongside ESF resources is declining in many Member States.
Given the more challenging labour market environment and the decline in public
spending in most Member States, it will be harder to achieve as good outputs and
outcomes from ESF funded interventions compared to pre-recessionary years. This
can be done, but it means that organisations in receipt of ESF funds will need to work
much more intelligently, be more innovative and create much more effective
partnership with other organisations able to support particularly more disadvantaged
individuals by deploying their mainstream funds. In effect, partnership working and the
alignment of ESF and mainstream funds need to become the norm for the remainder of
the 2007-2013 programming period, and these partnerships must be effective in
practical terms and involve more than key organisations signing statements of intent or
other forms of documentation.
Real challenges are posed for ESFfunded A2E interventions.
Reduced demand for labour makes it hard for A2E key areas of interventions to
get results.
This is particularly the case for more disadvantaged groups (often priorities for
A2E assistance).
Reacting to the Recession

In the report on Evaluation of the Reaction of the ESF to the Economic and Financial
Crisis6, the key changes highlighted in the EU labour markets include the following:
The net loss of 5 million jobs between Q3 in 2008 and Q3 in 2009.
Big variation in employment decline across Member States.
Hits young, women, vulnerable groups and those in part-time work seeking to
work full-time.
6 Evaluation of the Reaction of the ESF to the Economic and Financial Crisis (2012). Metis and Wiiw.
page 16
The Member States responded quickly to the economic crisis.

27 Member States deployed a 28% increase (201bn to 256bn) in expenditure
on Active Labour Market Policy.
New key areas of intervention introduced included short-time working
arrangements (STWA), temporary wage subsidies, reductions in non-wage
costs, public sector employment, assistance to become self-employed and
business start up, upgrading skills and promoting mobility.
In terms of the reactions of the ESF, the report noted the rational response to the crisis
by ESF. The traditional focus on skill development and increasing A2E were both seen
as coherent and rational long-term labour market strategies. However, the report
pointed up the need for changes in relation to the short-term labour market
environment, including the following recommendations.
Support crisis-relevant activities of national policies in a timely way.
Extend the approach of accompanying STWA with training.
Develop the ability to react quickly.
Phase out crisis-related measures that risk impeding longer term ALMP
investments in fighting high and long-lasting unemployment.
Support Anticipative measures in order to better avoid and combat future
crises, backed by tried and tested ALMP measures.
Strengthen the ESF focus on structural measures and vulnerable groups.
Support reforms in national ALMP so as to improve, upgrade and anticipate
future skill demands.
Support pilot projects aimed at finding more flexible forms of working and
responding to challenges relating to skills, qualifications and experience.
The study grouped Member States into six categories based on analysis of the impacts
of the crisis, approximately through to 2010, but also the response to the crisis in terms
of national labour market policies and the contribution of ESF.
Group A Strong economy where crisis had average impact, and no significant
labour market actions taken. All had extensive experience using ESF (BE, LU,
DK and NL).
Group B Crisis had an average impact and responses largely in terms of
national labour market policies, with ESF budget consequently small in relation
to GDP (AT, FR, DE, SE, FI, UK, and north of IT)
Group C Crisis had high impact on labour market and ESF funding was
considerable. ALMP medium share of GDP but with ESF major contributor.
Countries have lengthy experience of ESF (PT, ES, IE and south of IT).
Group D Countries with Convergence regions effected by crisis and with
capacity to use ESF as main ALMP instrument (PL, LT, LV and EE).
Group E Countries severely damaged by recession and experiencing
problems implementing ESF, including countries with already fragile economies
and still in an economic phase (RO, BG and GR).
Group F Growing economy pre-crisis and hit by crisis leading to difficulties in
implementing ESF. All had some (short) experience of ESF previously (SI, HU,
CZ and SK).
page 17
The analysis in this report underlines the challenge of assessing the effectiveness of
ESF A2E interventions in such a turbulent period for European economies. The
grouping of economies in terms of their labour market situations, their experience with
ESF and their capacity to respond to change highlights the challenge of drawing broad
comparisons across Member States within this analysis of ESF A2E.
Variations in Labour Market Impacts of Recession

Employment Across EU
The study of the reaction of ESF to the crisis came up with much valuable analysis as
well as an insightful grouping of Member States. For this report we have extended the
analysis modestly to look at employment and unemployment change using more
recent data. Our analysis shows that since the crisis, the countries in the EU27 exhibit
a wide spectrum of labour market results.
Figure 1 looks at changes in employment levels.
The Baltic States difficulties are shared with Greece, Bulgaria and Ireland, all of
whom have experienced substantial declines in employment.
In terms of Member States where employment levels have declined modestly,
there is a mix of Member States including both north and south Europe and
newer and well established EU members.
The small group of Member States showing strong employment growth is also
quite varied, with Poland in particular among larger economies riding the
recession well in terms of employment levels, but with smaller countries such as
Malta and Luxembourg also doing well.
Figure 1. % Change in Employment Levels (Aged 15-64), 2007 to 2011
Source: Eurostat Labour Force Survey

Note: Figures are annual averages.
page 18
Figure 2 provides an insight into one of the key elements of employment change
associated with the recession, namely that where there has been job recovery it has
been stronger for part-time employment. Particularly for the some of the newer
Member States, part-time employment growth has been substantial. Part-time
employment is a key labour market characteristic influencing what A2E measures are
able to achieve. The difficulty is that part-time employment generates a lower financial
incentive to move from welfare to work. The performance of the Polish economy is
interesting here as its employment levels have grown strongly while at the same time
part-time employment has fallen.
Figure 2. % Change in Part-Time Employment (Aged 15-64), 2007 to 2011

Note: Individuals describing their main job as part time.
Unemployment Across EU Member States

The change in unemployment over the same period clearly maps closely into the
employment changes with some variations for individual Member States. Although it is
possible to group Member States the unemployment experiences of each one tends to
sit on a spectrum
The Baltic countries have fared very badly in terms of unemployment with
massive increases in levels of unemployment but Ireland and Spain have also
been severely impacted.
In line with the employment changes Germany, Austria, Belgium and Poland
have fared relatively well over the 4 years since 2007.
page 19
Figure 3. % Change in Unemployment Levels (Aged 15-64), 2007 to 2011

Note: Figures are annual averages. Unemployed persons are those who are not employed, had actively
sought work during the past four weeks and are ready to begin working within two weeks.
Overview of Labour Market

The purpose of this short section is simply to demonstrate what is already well known,
i.e. there has been a significant variation across Member States in labour market
experiences through the course of the recession. The charts do illustrate, however, the
extent of the variation. This needs to be taken into account in considering how ESF
A2E has worked over the same period.
2.4 This Synthesis Report

Key Questions Addressed
The overall purpose is to attempt to answer a number of major questions in relation to
A2E activity within ESF.
What progress has been made in delivering A2E within ESF across the Member
States?
What have been the major key areas of intervention and which have been the
key priority groups for A2E?
What significant results have been achieved to date in relation to A2E?
In broad terms, what success factors can be identified as well as good practice
in implementation?
What is the Community Added Value of A2E activity to date?
What can be said about the data sources and their quality and reliability in
relation to answering key questions about the delivery and effectiveness of A2E?
page 20
Are the evaluation approaches and methodologies in place to allow for robust
evaluation of the results and impacts flowing from ESF funded A2E
interventions?
In addition, the report comments upon information sources and methodological issues
in relation to the evaluation effort.
Methodological Approach and Reporting Process

The broad approach was to work through a base resource of 27 Thematic country
reports and Context Reports. The purpose of the thematic country reports is to
analyse in depth themes most covered by the ESF evaluations and of most interest to
the Commission and to develop a fuller understanding of the results obtained from ESF
investment related to these themes through a review of evaluations undertaken in each
of the 27 Member States. In those Member States where no evaluations on A2E have
been published to date, Context Reports have been produced on the basis of the AIRs
and any other available information.
This report deals with the A2E theme.
A2E key areas of intervention are often ones where Member States often deliver
their own significant effort through Public Employment Services and Active
Labour Market Policy, including training activity of various kinds.
ESF will often add resources to support existing efforts, but will also generate
additional resources for key priority groups.
For countries with very limited mainstream activity around A2E, ESF can play
significant role in establishing and supporting A2E interventions.
Even in countries with well established A2E activity, ESF funding at the margins
can make a significant difference by helping bring some innovative approaches
into the mainstream and by prioritising groups such as Roma population.
The purpose of this synthesis report is to analyse the information in the country
reports to provide an overall picture of the achievements of the ESF in these fields and
the evaluation methods used across the EU. This is in keeping with the Commissions
remit to evaluate the effects generated by the money spent within the ESF.
Structure of Report
The report is structured in the following way to address the key questions which require
to be considered.
Overall Progress in Delivery of ESF Access to Employment. The next
section of the report looks at ESF A2E in an aggregate fashion across the 27
Member States and considers progress to date.
Analysis of Interventions. This section discusses the types of ESF A2E
interventions and the target groups across the 27 Member States and also pulls
together information on the effectiveness of interventions.
Factors Influencing and Good Practice in Implementation. Factors inhibiting
and promoting the performance of ESF A2E interventions are considered here,
and examples of good practice are collated.
Community Added Value. The Community Added Value (CAV) flowing from
ESF A2E activity in the Member States.
page 21
Evaluation Data, Approaches and Methodologies. This section provides a
concise assessment of the data and evaluation approaches used in coming to
judgements about how much ESF A2E activity is being delivered across the
Member States, and the extent to which this activity is effective.
Conclusions. The broad conclusions emerging from the study are brought
together in the final section of the report.
page 22
3 Overall progress in delivery of ESF Access to Employment
2007-2013 ESF Operational Programmes

75.95 billion has been allocated in 117 OPs across the 27 Member States for the
2007-2013 programming period. These OPs are drawn up at national/multiregional or
regional levels reflecting the differing constitutional and institutional backgrounds. They
provide information on the priorities chosen, the allocation of funds, the targets set and
the target groups approached. Table 1 shows the aggregate picture by Member States,
giving the total amount of ESF funds and the proportional amount by Member States
as well as the number of OPs under each of the objectives, differentiated by
national/multiregional and regional programmes. The countries with the highest
allocations are marked in red. Most Member States with large allocations have regional
ESF programmes.
Table 1. Number and Volume of ESF OP by Member State
Number of programmes
Convergence Competitiveness total
Total ESF Funds per together
Funds MS in % nat/ nat/ with
Country (Meuro) EU 27 mulitreg regional mulitreg converg. regional
AT 524 0,7% 1 1 2
BE 1.073 1,4% 1 1 4 6
BG 1.185 1,6% 2 2
CY 120 0,2% 1 1
CZ 3.775 5,0% 2 2 1 3
DE 9.381 12,4% 1 6 1 11 18
DK 255 0,3% 1 1
EE 392 0,5% 1 1
ES 8.057 10,6% 3 8 2 11 22
FI 619 0,8% 2 2
FR 5.395 7,1% 4 1 5
GR 4.364 5,7% 4 3 4
HU 3.629 4,8% 2 2 2
IE 375 0,5% 1 1
IT 6.938 9,1% 2 5 1 16 24
LI 1.028 1,4% 2 2
LU 25 0,0% 1 1
LV 551 0,7% 1 1
MT 112 0,1% 1 1
NL 830 1,1% 1 1
PL 9.707 12,8% 1 1
PT 6.512 8,6% 2 1 2 1 4
RO 3.684 4,9% 2 2
SE 692 0,9% 1 1
SI 756 1,0% 1 1
page 23
Number of programmes
Convergence Competitiveness total
SK 1.500 2,0% 2 2 2
UK 4.475 5,9% 3 1 3 6
EU 27 75.953 100,0% 29 29 12 47 117
Source: Metis compilations from ESF Data base (http://ec.europa.eu/social/home.jsp)
Overview of Financial and Physical Progress in Relation to ESF A2E

In broad terms, it is estimated that overall 30% of the funds for the 2007-2013
programming period were allocated to A2E interventions at the outset. However, from
2008 most economies across Europe and North America have experienced a severe
recession, with consequent major pressures on public finances. Both of these factors
can impact on the Structural Funds more generally, but also ESF A2E in particular
given that ESF A2E supports key areas of intervention close to the labour market as
discussed in Chapter 1.
Table 2 summarises the position on ESF A2E across the 27 Member States, from the
start of the programming period up to December 2010, based on the analysis
contained in the country reports.
In addition to available evaluation studies, in the majority of cases experts based
their analysis on AIR 2010. Due to the timing of the contract, AIR 2011
availability was limited as they were not yet public documents.
Evidence on progress in terms of number of recipients and the results they
achieve are typically based on monitoring information provided by the
beneficiary organisations. This raises issues about the consistency and
robustness of this information.
In many Member States the evaluation evidence is limited and often very
fragmented.
A major additional difficulty is that in considering the evidence summarised in this table
it is important to bear in mind that many experts across the 27 Member States were
required to estimate the A2E activity with some of the estimates subject to potentially
wide margins of error where A2E could not be readily equated with a specific PA or
other classification under which key data on ESF at the OP level are organised. A
fuller version of Table 2 can be found in Annex 1 providing more detail on this on a
country by country basis. However, a number of key points emerge from Table 2.
Good Volumes Overall

Table 2 shows that ESF A2E has generated significant levels of expenditure, activity
and results.
Around 23 billion in total public sector cost (Member State plus ESF) has been
committed to A2E.
A small number of Member States account for a high percentage of the financial
commitment to ESF A2E. Nearly 90% of the total public sector costs associated
with ESF A2E are committed in only 8 Member States DE, ES, FR, GR, IE, IT,
PL and UK. In part this reflects high Member State contributions relative to ESF
in percentage terms in a number of these countries, as with the co-financing of
page 24
the UKs national employment programmes which are major A2E interventions.
Additionally, this grouping contains some of the most populous Member States.
Over 12.5 million final recipients have been engaged through ESF A2E
operations. This may be an over-estimate to some extent as in some country
reports experts noted that there was a degree of double counting due to the way
management information was collected. Nonetheless this is a substantial
number of people and the programming period still has a number of years to run.
Turning to employment results, the ideal approach would involve an assessment of
two key measures.
Job entry: the numbers finding employment (or self-employment) subsequent to
completing their ESF A2E intervention.
Job sustainability: the numbers and percentages of those finding employment
who are still in employment, not necessarily with the same employer, say 12
months after first finding a job.
The results presented in Table 2 rely entirely on job entry as only one country report
provided evidence on job sustainability. The approach to measuring job entry in the
country reports fell into two categories.
Finding a job at an unspecified point in time after completing the A2E
intervention generally derived from monitoring data.
In a job either 3 or 6 months after completing the A2E intervention, typically
based on evaluation reports.
The employment results column of Table 2 shows that:
For only 3 countries, where the analysis is based on context reports, there is no
estimate for an employment result.
A number of country reports generated percentage employment result estimates
where absolute numbers are preferred.
Almost half of the country reports were not able to specify precisely the definition
of the employment result, but on the evidence of the other country reports the
high probability is that they all measure job entry defined in one of the two ways
set out above.
Bearing in mind the above, the best estimate is that 2.4 million employment results
have been achieved.
Factors making it more likely that this is an underestimate include the fact that
we have no absolute numbers for AT, BE, BG, FI, HU, LV, and PL. Additionally,
some of the results are for shortened time periods and in the great majority of
cases the results are only until the end of 2010.
Factors leading to overestimation include measurement issues for some
countries. For example, job entry in DK is measured alongside increasing
employability and competence levels; IE includes entering education and
training; SI includes some already employed at the start of their ESF
intervention.
page 25
Table 2. Financial and Physical Progress for ESF A2E, to December 2010
Member Public Sector Final Recipients Employment Result

State Cost Committed Achieved Target % Achieved Target %
(Meuro) Target Target
AT 128.2 65,578 70,000 93.7 34.9% 47.5% 73.4
BE 578.9 497,542 26-81%
BG 196.5 15,365 5,509
CY 37.9 2,979 5.9% 16.5% 35
CZ 143.2 268,776 642,500 41.8
DE 1,018.3 131,657 7,455
DK 30.3 14,939 17,000 88.0 2,424 17,300 14
EE 80.8 80,225 11,699
ES 6,179.1 4,379,545 6,533,824 67.0 947,201 2,978,730 31.8
FI 230.9 39,120 36.7%
FR 1,119.3 2,305,860 754,708
GR 676.3 373,525 N/A
HU 93.5 108,000 138,000 78.3 12.9% 28.5% 45.1%
IE 1,051.8 213,346 59,996
IT 1,711.8 311,400 21,250
LT 152.6 146,000 183,200 79.7 58,400 73,200 79.8
LU 3.4 2,882 1,506 191.4 104
LV 234.0 255,996 231,073 110.8 25-82% 30-40% 83-205%
MT 16.3 5,740 715
NL 74.1 33,176 21,180 156.6 13,965
PL 4,537.6 1,086,399 2,149,000 50.6 18-45% 30-58% 44-119
PT 229.4 64,854 124,088 52.3 13,367 11,600 115.2
RO 13.7 38,288 235,000 16.3 N/A
SE 227.9 40,175 75,000 53.6 N/A
SI 161.3 85,912 205,900 41.7 56,700
SK 143.2 260,030
UK 4,163.0 1,754,055 1,223,900 143.3 449,846 306,800 146.6
EU Total 23,233.3 12,581,364 18,851,171 93.7 2,403,339

Note: No data or aggregated data available, typically context reports.
Found job post-ESF intervention
Lack of information on specific definition of employment result.
Jobs created only, not A2E job entry
page 26
Too Much Missing Information

Moving from the left to the right in Table 2 empty cells are increasingly likely to be
encountered although these partly relate to whether or not target levels had been set
by Member States for A2E within their OPs.
However, for some countries only an estimated percentage job entry rate was available
with no figure for the absolute number of final recipients enjoying an employment
result. It is critically important to be able to demonstrate in simple absolute numerical
terms whether ESF A2E operations have delivered effectively relative to the budget
deployed so more attention needs to be paid by Member States to ensuring that they
are able to generate this basic information on employment results.
Big Variations Across Member States

There is a high degree of variability in the extent to which targets have been met for
activity carried out by end 2010, the most common reporting period. The range of
explanations for variability tends to fall into the following categories.
Where activity is well below target in terms of recipients engaged, this is
generally put down to slow starts in 2007 and 2008 but with little explanation for
the reasons for this. In some of these country reports evidence is produced to
show an acceleration of activity.
Where activity is above target, this is generally attributed to the impact of the
economic recession which led a number of MAs to revise their OPs to place
more emphasis on helping unemployed people whose numbers were growing
rapidly. Evidence on employment results suggests that in some country reports,
employment results are well above target and are explained in terms of client
group characteristics such as working with shorter term unemployed or with
unemployed graduates. For example, the number of recipients and the number
of results well above target in the UK reflect co-financing with the Department for
Work and Pensions, the state body responsible for the unemployed and these
have risen significantly in scale during the recession.
For many countries there appeared to be no target in relation to the number of final
recipients and/or employment results. This may simply reflect the difficulty in some
countries in disaggregating targets relating to A2E activity sitting within OPs covering a
wider range of interventions. However, where targets have been reported the following
can be observed.
For a number of countries (AT, DK, LU, NL and UK) targets for final recipients
are either close to being met or have already been met.
For a number of other countries (ES, HU, LT) progress towards the target for
final recipients is in the 60-80% range.
The remaining countries for whom there are targets are less than 60% of the
way there namely CZ, PL, PT, RO, and SE. In the case of RO the target is only
16.3% achieved.
In relation to employment results AT, LT, PT and UK are either making good
progress or have already achieved target. However, the England OP at 16% into
jobs is still below the 22% target for the proportion of participants going into jobs.
It is dangerous to be too strong in conclusions here, particularly as there may be
issues with the target setting process across Member States. For example, the UK is
page 27
nearly 50% ahead of target for employment results, but the expert argues this largely
reflects co-financing of the major public programmes for the unemployed - whose
numbers have been swollen by the recession. The expert goes on to argue, particularly
in relation to the final recipients target, that numbers well ahead of target could equate
to a diluted intervention. Although this may reach more people, it is less likely to
achieve sustainable job entries, particularly given the challenging nature of the labour
market.
Scope for Understanding Performance Variation . . . But Not Yet!

There is scope to try to explain variations in ESF A2E performance to date across
Member States. For example the architecture of ESF in terms of number of OPs and
the nature of national and regional structures could have an influence if complexity of
architecture creates barriers. It is interesting to note that some Member States with a
single OP (AT, LT and NL) show good progress against target for number of recipients
and number of employment results. However other countries (UK and PT) also have
made good progress despite having a larger number of OPs. It may simply be that
greater complexity of architecture is compensated for by greater flexibility and
appropriateness in design.
More could be done here looking at features such as ESF as a percentage of total A2E
funding, labour market characteristics of groups of economies, etc. At this stage
however the difficulties are:
Too much missing information on performance at the aggregate level for
individual Member States.
Uncertainty about the accuracy of the measures.
Variations across the Member States in the way targets are being set.
Moving forward, however, there is value in exploring the development of a better
quality explanatory framework for assessing performance across Member States.
page 28
4 Analysis of interventions and target groups
4.1 General analysis

The evidence of the country reports prepared by the experts is that for some Member
States information tends not to be available on interventions (by which we mean
specific delivery such as vocational training programmes or employment subsidies) but
rather around PAs. This is a reflection of the fact that evaluations are largely
commissioned with a view to providing management information at OP level with
limited analysis below the level of PAs. However, some more focused evaluations were
available on the detail of A2E interventions, and this section builds on these.
A key broad emerging finding has been the evidence of additional resource being
allocated or re-allocated to PAs, and activities from around 2009 addressing the issues
of:
Rapidly rising levels of unemployment, and so focussing more resources on
A2E, including increased support for self employment and enterprise starts.
Measures to stimulate employment demand in private sector enterprises and/or
develop intermediate labour markets, social firms, etc.
The great difficulty from a reporting standpoint is that the information for this sub-
section is often not tabulated and sometimes verges on the anecdotal.
The re-allocation of resources in relation to such a major economic crisis, with
increased focus on A2E interventions, is a positive feature of the management of ESF
across and within Member States. Given the length of the programming periods, undue
rigidity in sticking to original delivery plans almost certainly means less value created
from the resources devoted if the concentration is on yesterdays problems rather than
current realities.
4.2 A2E interventions and target groups

A2E interventions
A wide range of interventions emerged from the country reports. These can be
grouped under three broad headings.
Enhancing Recipient Employability
The taxonomy here ties into the employability pathway concept outlined in a previous
chapter.
Initial assessment work with recipients to identify what supports they need in
order to move them towards and into employment, which they can subsequently
sustain. There is less visibility of these types of interventions in the country
reports probably because of its detailed nature but this may reflect a weakness
in the structure of interventions.
Providing information, advice and guidance to recipients to help them make
the right decision and take advantage of the right interventions. A number of
country reports refer to these types of supports in relation to information about
opportunities in the labour market and around relevant training opportunities, for
example.
Helping with the personal development of recipients, in terms of raising
confidence and aspirations and generally trying to make people more resilient.
page 29
This is an intervention common to many of the Member States on the basis of
the CR evidence.
Addressing basic and core/key/employability skills. Basic skills typically
include literacy, numeracy, IT and basic language skills particularly for migrants.
Employability skills include the things that employers generally demand such as
reliability in attendance, capability for working in teams, etc. The country reports
suggest that ESF A2E contributes significantly to the former but less to the latter.
Creating new or upgraded vocational skills are a traditional feature of ESF and
remain a significant component of A2E within ESF. The economic recession
appears to have given a stimulus to this with substantial redundancies in jobs
and skills generating a significant demand for re-training.
Another common intervention is work experience/internships. These perform
a number of functions as articulated in some of the country reports.
- They help generate the employability skills and general workplace awareness
discussed above.
- They provide an experience which can feature on a CV in relation to job
search.
- They offer a direct opportunity to impress an employer and so secure a
temporary contract or permanent employment.
Graduate internships can clearly be seen as an intervention in a number of the
Member States but the work experience intervention more generally has been a
significant response to the rapidly deteriorating youth employment situation across
Europe.
Support for enhanced skills in job search, job application and interview
processes. Again this is a traditional intervention for unemployed recipients of
ESF A2E resources. It is also very common across the Member States.
Post job entry support, sometimes called aftercare, is mentioned in a smaller
number of the country reports. The rationales for this generally relate to the
following issues:
- Particularly in a context of labour market depression, jobs available are often
on a short term and temporary basis. Recipients of A2E interventions who
succeed in finding a job may also need help in keeping the job or moving
quickly to another job.
- Particularly for recipients who fall into the most disadvantaged categories and
confront Social Exclusion, they may have a higher degree of fragility in the
early stages of a new job and both the recipient and their employer can
benefit from aftercare services.
In broad terms for ESF, this type of intervention is a means of securing longer term
value from the investment already committed to get the recipient into the job in the first
place.
Supporting Creation and Retention of Employment
ESF A2E resources in a number of Member States are committed to helping
unemployed individuals into self employment and enterprise start up. This is
a recognition of the fact that self employment is a significant component of total
employment in many Member States.
page 30
ESF is also deployed in a number of Member States to incentivise employers

to recruit and/or skill target groups of recipients, such as young unemployed.
Again this is a traditional use of ESF funding which appears to have become
used more commonly again due to the depressed nature of the labour market
and the low rates of recruitment associated with this.
There are related actions involving temporary income support which is
effectively subsidising training for workers as an alternative to them being
declared redundant.
Particularly in areas of severe labour market depression and for groups who are
very long term unemployed and multiply disadvantaged, ESF A2E support is
being provided in a number of Member States for the creation of transitional
employment or supported employment. The intermediate labour market
model is a common form of transitional employment and the supported
employment model which involves modifying workplace environments and
providing much more intensive in-work supports is another traditional form
targeted at significantly disadvantaged individuals such as young people with
learning disabilities.
Capacity Building
In some of the Member States newer to the EU, ESF A2E money has been
committed to building the capacity of public employment and training services,
including the creation of more effective systems and processes. The relevance
of ESF investment here is that it raises the return on the investment of ESF A2E
money in basic skills, training and other interventions as the public employment
service will often then be tasked with finding employment for the recipients.
Table 3 describes the A2E interventions delivered across the Member States in a
simple fashion. Insufficient information was available to quantify the activity across the
countries, or indeed within specific countries, in terms of allocated spend. It also needs
to be noted that the evaluations available do not always reflect the full range of A2E
interventions supported by ESF in individual Member States, although gaps in
information are to some extent compensated for by the use of AIRs.
There are some broad findings.
Vocational skills development is evident in at least 22 Member States. This is
very much a traditional ESF A2E activity.
Creating the employability skills (reliability, team working, etc) required in the
workplace more generally is featured in at least 16 Member States. This reflects
a growing appreciation, particularly over the last decade, that employers want
much more from their recruits than the technical skill required to carry out the
job. Investing in these skills also makes sense when there is uncertainty as to
which sectors will grow most when economies recover from recession.
Childcare and family support are recorded as A2E interventions in 16 Member
States, in part due to the prioritisation of women within ESF A2E and in
recognition of the major barrier associated with childcare availability and cost.
Work experience and internships appear as A2E interventions in 15 countries.
From the commentaries in the country reports these have become more
common interventions with the onset of the recession, and in a number of
countries graduate programmes have been developed.
page 31
Support for self employment and enterprise startup is provided by ESF A2E
in 14 countries. In some countries this has been introduced as a result of the
recession to compensate in part for the decline in employment in existing
businesses and organisations. Some country experts have argued that as a
consequence of the recession it is extremely difficult for unemployed and other
people to make a sustainable start in business.
Employment subsidies are supported by ESF A2E in at least 11 countries, and
again are seen by some as a counter-recessionary intervention which are the
margins can stimulate in particular small businesses to expand employment but
also as a measure to make it more likely that an individual from a prioritised
group (such as the young unemployed) will obtain an employment opportunity in
an increasingly competitive labour market.
Labour market advice and guidance features in at least 10 of the Member
States with support from ESF A2E. This is increasingly important in the context
of significant labour market change and it is perhaps disappointing not to see
more activity in this sphere.
Capacity development of, in the main, national, public and training services
receives ESF A2E support in at least 9 countries. As expected these are
principally more recent members of the EU although FR and IT are also
recipients of this support.
Some types of intervention appear not to be all that prevalent in terms of receipt of
funds from ESF A2E.
Support for personal development and confidence building is mentioned in
at least 5 country reports, but this seems low given the focus of ESF A2E often
on longer term unemployed and more disadvantaged groups.
Support to find work is reported for at least 7 countries but this should be
central to any concerted approach to A2E.
Support to sustain work (i.e. post job entry support) appears in only 2 country
reports, and yet there are significant issues about sustainability of employment
secured and this becomes a greater problem in a more depressed and flexible
labour market with short-term contracting a more significant form of employment.
In exploring variations across Member States in terms of the types of ESF funded A2E
interventions, the likely broad influences on what happens in an individual country are
the existing mainstream A2E delivery (active policy) and the social security system
(passive policy). Although it is beyond the scope of this study to consider in detail how
these impact, two broad points can be made in relation to the information in Table 3.
The relative importance of capacity building of public institutions in relatively new
EU Member States has already been identified, and is confirmed in the tabular
analysis.
Beyond this, the pattern of A2E activity does not appear to vary in any obvious
way by the groupings of Member States. This is perhaps surprising as the labour
market circumstances and Member State mainstream services in relation to A2E
are likely to vary significantly. This raises interesting questions about the
planning of OPs and the detailing of delivery that flows from these. The most
obvious issue is the extent to which those involved in planning an OP take
account of complementary services, such as health and social care, when
building the content of their programme.
page 32
Draft A2E Synthesis Report
Table 3. Types of intervention

MS Assess Careers Personal Deving Vocation- Work ILMs/ Employ- Support Support Childcare Post job Support Training Inst. Others
-ment and devment key al skills exp./ social ment for self- to find and entry for up- for dev.
of A2E labour and employ- devment intern- firms subsidies employ- work, job family support skilling existing (PES,
needs market confidence ability ships ment/ search/ support (prog- employee etc.)
info., building skills for start-up interview ression) s
advice work- skills
and place
guidance
AT
BE
BG
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HU
IE
IT
LT
LU
LV
MT
page 33
MS Assess Careers Personal Deving Vocation- Work ILMs/ Employ- Support Support Childcare Post job Support Training Inst. Others
-ment and devment key al skills exp./ social ment for self- to find and entry for up- for dev.
of A2E labour and employ- devment intern- firms subsidies employ- work, job family support skilling existing (PES,
needs market confidence ability ships ment/ search/ support (prog- employee etc.)
info., building skills for start-up interview ression) s
advice work- skills
and place
guidance
NL
PL
PT
RO
SE
SI
SK
UK
EU 8 10 6 17 23 15 6 11 14 7 17 3 3 9 9 13
27
Notes:
1. EU-12 countries shaded in blue.
2. Czech Republic, Denmark, Luxembourg, Romania, Sweden and Slovak Republic either no information on interventions provided in country report or unclear.
3. Childcare and family support 8 specifically note childcare as support; 9 implied.
4. Others include apprenticeships; education (including youth education, further education and higher education) and assistance to gain qualifications; rehabilitation
services; provision of help/assistance services for disabled individuals; adaptions to workplaces; research/PhD grants/scholarships; accreditation of existing skills and
competencies; grants for travel/equipment; support to increase geographic mobility.
page 34
Target groups for A2E

It is common to find for ESF A2E interventions, that the target groups are similar to
those for Social Inclusion. The principal target groups across most Member States
include:
Women.
Ethnic minorities.
Disabled people.
Young people generally.
Young people specifically those not, or at risk of not being in education,
employment or training.
Older unemployed people.
Long term unemployed people, generally 12 months or more out of work.
People with multiple disadvantages and complex needs.
Migrants and asylum seekers.
Although there is some variation, most Member States prioritise most of these groups.
There is relatively limited evidence for a number of Member States on the extent to
which participation levels for these groups are being achieved, which is a critical issue
for the effectiveness of prioritised approaches.
Table 4 summarises the position across Member States in relation to target groups.
Some of the key points emerging from this table are as follows.
Young people are prioritised in at least 19 countries. Although the definition of
young varies typically this is defined as under 25. A small number of countries
(AT, BE, DK and UK) additionally prioritise young people not in education,
employment or training (NEET) or at risk of becoming NEET.
Women are a key priority for ESF A2E for at least 17 Member States.
Unemployed are prioritised in 18 Member States with 14 targeting long-term
unemployed, but it is safe to assume that in virtually all Member States using
ESF to fund A2E interventions the unemployed are likely to be a priority group.
People with multiple disadvantages and complex needs are an ESF A2E
priority in 15 Member States at least.
Older people generally unemployed or inactive, or at risk of unemployment, are
prioritised in at least 14 countries.
Migrants and related groups are prioritised across 11 Member States. For 3
countries (HU, LT and PL) the Roma population are a specific priority.
Reflecting the impact of the recession, employees at risk of losing their jobs are
a priority in at least 10 countries.
There appear to be some variations in prioritisation across groups of countries.
Disabled people are more likely to be targeted by ESF measures in the
countries that are newer to the EU, possibly reflecting less good quality
mainstream services for the disabled in these Member States.
Migrant workers are more likely to be a target group in older EU countries
reflecting the migration patterns favouring larger or more developed economies.
page 35
Table 4. Target Groups
MS Unemploy- Long term Existing Women Young Young Older Graduates Ethnic Migrants/ Roma Disabled People Others
ed unemploy- employees people people people minorities refugees/ people with
ed at risk NEET or at asylum multiple
risk of seekers disadv.
NEET and
complex
needs
AT
BE
BG
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HU
IE
IT
LT
LU
LV
MT
page 36
NEET and
complex
needs
NL
PL
PT
RO
SE
SI
SK
UK
EU 27 18 14 10 17 19 4 14 5 5 11 3 12 15 14
Notes:
1. EU-12 countries shaded in blue.
2. Others include persons with low qualifications/skills; prisoners and ex-offenders; young people in/leaving care; parents returning from maternity/paternity leave or with
caring responsibilities; families of disabled persons; war veterans; persons engaged in subsistence agriculture; men; students; and over 30s.
3. Country reports for Spain and Finland contain no or limited information on target groups.
4. The Maltese country report presents data on the percentage of final recipients in each of these groups but there is no evidence they were the OP stated priorities.
page 37
4.3 Significant results achieved to date
Clearly the purpose of ESF-funded A2E is not simply to provide useful activity to final
recipients. The resources there are to produce results for individual recipients, but
also of course raise the capacity of public and other organisations working to support
these recipients. In crude terms the results can be categorised in two very broad ways.
Measurable progress towards sustainable employment in terms of increased
skill levels and qualifications, growth in self-confidence, etc.
Sustainable employment.
As indicated earlier, the evidence to date is largely about entry to employment rather
than sustainability of employment secured. Additionally, the quantitative evidence from
the country reports on progress towards employment is virtually non-existent.
Overall Results
Before discussing the employment results there are some general problems with the
data which need to be stated at the outset.
In most instances the data are derived from AIR 2010, and consequently are
principally based on monitoring data generated by beneficiary organisations.
The employment results are generally expressed as a percentage of final
recipients rather than as a percentage of a cohort of recipients who have
completed their ESF funded activity. This means that as programmes are
building up and the number of recipients is increasing percentage employment
results are artificially deflated, and cost per result inflated.
Referring back to the final set of columns in Table 2, a number of conclusions can be
reached on employment results.
In very broad terms, employment results in most are at or close to target levels.
On the downside, there is a suggestion by a number of experts that the initial
targets were notional and not particularly stretching.
On the upside, the targets were set prior to the worst economic recession for 30
years which must act to lower the chances of A2E recipients finding and
sustaining work.
Bearing in mind the earlier caveats on the measurement of employment results, Table
5 indicates that:
The spread of percentage employment results is very wide and the
corresponding variations in cost per result is similarly extensive.
The relatively low percentage achieving employment results in DE, IT, LU, etc. is
almost certainly a reflection of the failure to measure what happens to a cohort
of recipients going all the way through the intervention to completion and
beyond, but will also reflect the rate of expansion of numbers over the
programming period.
In some cases (BE, LP and PL) there is no aggregate employment result for the
Member State but a range of results for different key areas of intervention can
be reported. These demonstrate the variation within a country across different
approaches can be as great as the variation across countries in aggregate
terms.
page 38
Table 5. Overall Effectiveness of ESF A2E

% Recipients Achieving Cost () Per
Employment Result Employment Result4
Austria (AT) 34.9
Belgium (BE) 26.0-81.0
1
Bulgaria (BG) 35.9 35,669
Cyprus (CY) 16.5
Czech Republic (CZ)3
Germany (DE)1 5.7 136,593
1
Denmark (DK) 16.2 12,500
Estonia (EE)1 14.6 6,907
1
Spain (ES) 21.6 6,524
Finland (FI) 36.7
France (FR)1 32.7 1,483
Greece (GR) No data at all
Hungary (HU) 12.9
2
Ireland (IE) 28.1 13,649
Italy (IT)1 6.8 80,555
1
Lithuania (LT) 40.0 2,613
1
Luxembourg (LU) 3.6 32,692
Latvia (LV) 25.0-82.0
Malta (MT)1 12.5 22,797
1
Netherlands (NL) 42.1 5,306
Poland (PL) 18.0-45.0
Portugal (PT)1 20.6 17,162
Romania (RO) No agg data
Sweden (SE) No data at all
Slovenia (SI)1 66.0 2,845
3
Slovak Republic (SK)
United Kingdom (UK)1 25.6 9,254
Note: 1. Calculated by authors using data from Table 2 (i.e. % Recipients Achieving Employment Result
= Employment Results Achieved / Final Recipients Received; Cost per Employment Result =
Public Sector Cost Committed / Employment Results Achieved). The other results are drawn
directly from the country reports.
2. IE Country report provided Cost Per Employment result. % Recipients Achieving Employment
Results calculated by authors using formula outlined in Note 1.
3. CZ and SK Data only on jobs created by beneficiary organisations.
4. Apart from the 5 country reports with no employment results data, the absence of figures on
cost per employment result reflects the position for those country reports where the employment
result was presented in percentage terms.
page 39
It is disappointing not to be able to present statistical information on progress towards
employment. This is surprising as the concept of measuring distance travelled is well
established in thinking about A2E interventions.
Results by type of intervention and target groups

Some of the country reports in the body of the text identify variations in employment
results across key areas of intervention and PAs. Table 6 has been put together from
the country reports to try and capture some of the variations within countries both
across key areas of intervention and priority groups. Below we attempt to summarise
the key findings based on a combination of the assessments by the experts in their
country reports and the statistical evidence collated in Table 6.
Results by type of intervention
It is difficult to generalise on this as there appears to be some variation across Member
States for given interventions. However:
Interventions closer to employers tend to achieve better results (at least in
broad terms), including work experience placements and internships. Clearly this
is likely to be associated with the more employable characteristics of the
recipients involved.
Related to the above, there are good results for apprenticeship training
focussed on younger people.
Work experience and internships tend to produce good results, but in some
countries (for example LT and FI) these results are skewed upwards as the
interventions are targeted at graduates.
There is some evidence that vocational training programmes are more
effective where employers have had a design involvement.
There is varied feedback on the effectiveness of support to help unemployed
people into self employment.
Results for employment subsidies vary quite markedly across countries with
particularly good results in some countries (NL and SI).
Some of the softer skills and personal development interventions have
relatively poor employment results, but this reflects the fact that they are
delivered typically at an early part of the employability pathway and tend to be
targeted more at people with multiple disadvantages.
Results by target groups
Results tend to be better for those closer to the labour market, for obvious
reasons. A significant number of country reports note that for those further from
the labour market, particularly given the crisis in European labour markets, a
more intensive approach to intervention is required but this tends to be more
costly to deliver.
In general terms, disabled people and older workers tend to have poor
employment results across Member States where statistical evidence is
available.
Results are typically poor also for clients with multiple disadvantages.
People resident in rural areas experience good results from their ESF A2E
intervention in PL.
page 40
Table 6. Effectiveness of A2E Interventions by Type and Target Group Selected

Member States
% Achieving Cost Per
Employment Employment
Result Result
Austria 34.9
Apprenticeships (PA2) 74
SOB/GBPs (PA2) Generally <20
Production schools (PA2) 30
PA3b 9 ( first labour
market)
15 (second
labour market
transitional
employment)
Young people (PA2) 67
Males 59
Females 71
Production schools (PA2)
With school leaving certificate 35
Without school leaving certificate 23
Belgium 26.0-81.0
Vocational training (BE-VL) 32
Job training and coaching (BE-VL) 39.8
Females (BE-B) 38
Under 25s 37.6
Estonia 14.6 6,907
Measure 1.3.1 Increasing availability of qualified labour 24
force
Measure 1.3.3 Active labour market measures 40
supporting welfare measures
page 41
Result Result
Finland 36.7
Employment subsidy to state, employment to ELY 28.8
(Regional MEE office)
Wage subsidy total 24.1
Apprenticeship total 59.2
Start-up money for companies 69.0
Employment benefit in labour market coaching 20.3
Subsidised employment total 36.4
Vocational labour market training 28.3
Total subsidised employment + vocational LM training 26.1
Coaching labour market training 16.5
France 32.7 1,483
Local employment and inclusion plans 34
Job coaching in Seine Maritime 37
Ireland 28.1 13,649
Labour market programmes 49 48,948
Labour market activation fund 29
(employment)
13 (self-empl.)
Specific skills training 58
Traineeship 72
Community Training Centres 41
Lithuania 40.0 2,613
ALMP Assistance to acquire professional skills 36
ALMP Employment subsidies 30
ALMP Public works 14
ALMP Vocational training 18
Social vulnerable/social risk group 30-35
Luxembourg 3.6 32,692
1.1 Increase employment rate of older workers 86 16,315
1.2 Increase womens employment rate 49 27,096
1.3 Facilitate insertion of young people in the labour 5 54,523
market
1.4 Activate persons with a large distance to the labour 45 29,425
market
page 42

Result Result
Latvia 25.0-82.0
Training of unemployed and job seekers Phases 1 and 2 21.8
Training of unemployed and job seekers Phase 3 95
(training and internships for unemployed graduates 25
and under)
Malta 12.5 22,797
PA3 (Promoting an equal and inclusive labour markets) 12.5
interventions and projects
Training and support for women 0
Support for vulnerable groups 11
Employment aid scheme (wage subsidy to employers to 94 14,395
incentivise recruitment from disadvantaged/disabled
groups)
Netherlands 42.1 5,306
Action A (projects that equip people with large distance to 5
labour market with skills and increase prospects for
sustainable integration)
Action B (projects that improve the position of prisoners, 16
people on a hospital order and juvenile delinquents on the
labour market)
Action C (programmes that promote the inclusion into the 71.4
labour market of students enrolled in professional
education and special secondary education)
Poland 18.0-45.0
Priority I, SO 2: To broaden the impact of ALMP 20.8
Priority VI, SO1: Increasing coverage of Labour Market 45.3 (employed)
Active Policy 19.8 (self-empl.)
Priority VI, SO2: Increasing employment among young 39.7 (employed)
people 8.36 (self-empl.)
Priority VI, SO3: Reduction of unemployment amongst
people disadvantaged in labour market
Women N/A
LTU 31.4
Persons with disabilities 33.1
Unemployed living in rural areas 41.8
Priority VI, SO4: 35.6 (employed)
Higher employment among older people 18.8 (self-empl.)
Priority VII, SO1: Better access to labour market for 17.6
people threatened by social exclusion
page 43
Result Result
Slovenia 66.0 2,845
Job subsidies 60.0
Developing new employment opportunities 63 (including
employed and
self-employed)
Unemployed persons 55
Persons already employed 100
Overview
Although Table 6 is able to present the results in only a piecemeal way, this way of
looking at results across both key areas of intervention and priority groups has great
potential to indicate where ESF A2E is being deployed effectively or not as the case
may be. Conclusions on this would of course need to be qualified by considering also
the cost of delivery and the degree of disadvantage experienced by the final recipients
of the intervention.
4.4 Effectiveness of ESF A2E interventions

Overall effectiveness
The broad findings on overall effectiveness are as follows.
Most country reports conclude with experts comments that ESF supported A2E
interventions are very effective, but the statistical evidence tends to be less
conclusive.
There are very few impact studies with robust methods, according to the view of
the various experts, and these would be needed to underpin a judgement of
overall cost effectiveness.
Where deadweight is reported on the basis of evaluation studies, or judgments
on deadweight have been made based on theoretical perspectives, some doubts
are raised about both overall effectiveness and the effectiveness of specific
interventions. However, very few country reports discuss deadweight.
A significant number of country reports note the need for more intensive and
costly interventions to raise the employability of more disadvantaged
participants, as a result of the recession in the labour market which has left
many of these people at the end of a much longer queue for employment
opportunities.
Effectiveness of specific types of interventions

As indicated above, there is some evidence on the results achieved by different types
of interventions and for different client groups. However, this is rarely correlated with
cost information and evidence on deadweight is extremely limited across the country
reports. The evidence as it stands is not sufficiently comprehensive, nor probably as
robust as required, to provide compelling guidance on what works best for whom. The
requirement here is for more data at the micro level which will allow evaluators to
page 44
analyse statistically with relatively large samples the results for different key areas of
intervention , while controlling for the characteristics of the final recipients.
page 45
5 Factors influencing and good practice in implementation
With such a major effort going into supporting A2E from ESF resources across
Member States, a key task for the experts analysing the experiences in their countries
was to identify the factors that:
Constrained the effective implementation of A2E key areas of intervention.
Supported the implementation of these interventions.
Although not all country reports were able to draw on evaluations with this degree of
detail, many were and below the key findings are captured. Building out from the
analysis, evidence on good practice was also collected, and this is discussed later in
this section.
5.1 Factors inhibiting effective implementation
Effects of Economic Crisis

The main factor inhibiting the performance of A2E interventions cited in most country
reports is quite simply the economic recession and the collapse of labour demand
across Europe. By definition, a programme tasked with getting people into jobs has to
work much harder and/or more effectively and/or with much more resource in
recessionary times to maintain a given level of employment result. The box below
illustrates some of the feedback from individual country reports.
Effects of Economic Crisis

Germany
Economic crisis made employers more reluctant to invest in or recruit more
marginalised groups in the labour market.
Hungary
A2E interventions were designed before the financial crisis and implementation could
not cope with collapse of labour demand. This is confirmed by mid-term evaluations.
General approaches and regulations do not cope well with big regional variations in
the impacts of the economic crisis, plus differences in concentrations of particularly
disadvantaged groups such as Roma minority.
Netherlands
Economic crisis is having a major impact on performance of A2E activities,
particularly for those in a disadvantaged position in the labour market.
Some key disadvantaged client groups are difficult to reach and motivate.
Spain
The crisis in the economy has impacted significantly leading to higher levels of
participation in ESF A2E than expected.
Financial problems in Spains public sector are making the co-financing of ESF
challenging.
United Kingdom
The wider economic environment is constraining progress towards targets. ESF
2007-2013 was designed to make inroads into worklessness on the back of a buoyant
labour market.
Depressed labour market conditions make it increasingly difficult to place ex-
offenders in employment.
page 46
Management and Administration of ESF

Notwithstanding the gravity of the crisis there were many familiar concerns and
complaints about the administrative complexity and burdens associated with ESF more
generally.
A number of administrative problems were raised in the country reports such as
the barrier associated with the volume and complexity of financial administration
of ESF leading to smaller and more potentially innovative providers declining to
engage.
Where management and delivery of programmes depend on collaboration
between a number of national, regional and local players the complexity arising
can constrain effective implementation.
Both veteran ESF Member States and relative newcomers appear to have
problems with management and administration issues.
Management and Administration of ESF
Germany
Issues relating to administrative management and implementation.
- Fragmentation of ESF in the light of increased decentralisation of implementation
leads to large number of programmes with small or average budget for each
but same administrative requirements. With increased number of programmes
this means more capacity building required for delivery agents.
- Administrative cost is taking up a higher share of budget, reducing money for final
recipients.
- Difficult to coordinate across a large number of agencies.
Hungary
Many concerns about administrative burden ESF places on beneficiary organisations.
This is exacerbated by frequent rule changes.
Italy
Low demand for evaluation and lack of urgency in commissioning/organising these
mean that good quality feedback on intervention effectiveness is not available.
Luxembourg
Too many administrative tasks.
Complicated database and guide.
Difficulty in finding co-finance for some target groups.
Issues with stability and capacity of beneficiary organisations.
Malta
Piecemeal planning and implementation of OP.
Lack of internal controls on effectiveness of management.
Lack of internal and external evaluations.
Burdensome and bureaucratic procedures making for costly management process,
particularly damaging given small size of OP in absolute terms.
Netherlands
Administrative burden associated with ESF leads to:
- More effort on administrative issues and less on getting results.
- Due to lack of experience with ESF admin and problems with admin data, claims
lie well below actual results with the latter remaining unreported and there is a
financial burden on the beneficiary organisations.
- Communications from the Commission are viewed as inconsistent and delivered
late with too many changes in the rules of the game.
page 47
Management and Administration of ESF (cont.)
Romania
There is no real basis for evaluating impact and no serious results are expected
earlier than 2013 by which time the programme is finished. The absence of the
necessary information makes it difficult to manage the programme effectively.
SOP HRD is a large programme with 7 PAs and 21 KAIs. It has national coverage
and 11 IBs (8 at regional level and 3 at national level). Many of the staff are new to
the system, are having to work on new procedures, taking on new ways of doing
business. Despite investment in training and capacity building there has been a high
level of turnover within the MAs and the IBs.
There are serious issues around capacity of beneficiaries to access and implement
funds.
Slovak Republic
Problems around clarity of definition of indicators for results.
Lack of any serious qualitative analysis of implementation by responsible authorities.
Limited involvement of stakeholders in implementation of national programmes.
Limited cooperation by responsible agencies in international networks, leading to a
failure to bring in experience from abroad.
Slovenia
Lengthy procedures in selecting beneficiary organisations.
Long process for making claims for payment.
Problems with monitoring information systems.
Spain
Administrative and institutional arrangements have introduced severe complications
leading to late execution.
Intervention Design Issues

In some instances the constraint on effective implementation was down to initial design
of the intervention. A number of different types of issue were raised here.
In some instances measures intended to be in support of A2E appear to have
become more by way of social interventions in the sense that the purpose is not
to help a final recipient secure an employment result but to provide some form of
activity as an alternative to unemployment.
There were issues around priority groups more generally as to whether sufficient
analysis had been done to understand the circumstances of a particular group in
order to design an intervention that would attract, retain and progress individuals
from the group.
Some examples are presented in the box below.
Intervention Design Issues

Hungary
In the design of A2E activities there has been some underestimation of the
importance of other factors such as structure of labour demand and the small margin
between benefits and accessible wages.
Lithuania
Need to select the most motivated target groups and to design the activity around
them.
Need to take more account of the economic recession and introduce measures which
tackle short term unemployment.
page 48
Portugal
Some evidence that the development of the social employment market has been
done in such a way that it becomes a social intervention rather than economic
integration into the labour market and so does not support the participation of the
vulnerable population in the labour market.
United Kingdom
Poor quality of targeting in relation to DWP and JobCentre Plus, with extensive
resource appearing to go to shorter term unemployed people more able to find work
on their own.
5.2 Factors contributing to effective implementation

A number of country reports contain evidence on factors that contribute to effective
implementation. The kinds of effects captured in the country reports include the
following.
An effective public employment service can contribute significantly to the ability
of ESF A2E interventions achieving good employment results. ESF A2E
interventions will often raise an individuals employability but the PES has a key
role to play in finding them sustainable employment.
Interventions delivered through national or regional operational programmes can
reach in particular more disadvantaged recipients by working in partnership with
organisations already engaged with particular groups of individuals or specific
geographical communities.
The creation of networks at the regional or national level which bring together
staff involved in delivering A2E can generate significant learning and process
improvements - as well as feeding into finding a more effective design for the
intervention.
Below some more detailed examples of success factors are articulated.
Responding Quickly and Flexibly to Change

Given the severity of the economic crisis speed and appropriateness of response can
contribute greatly to the effect of the implementation of ESF A2E. Some examples are
boxed below.
Responding Quickly and Flexibly to Change
Denmark
Denmark starts with identifying needs for adjusting at the regional scale to changing
conditions in local labour markets. This requires close cooperation with regional
partners to understand the impact of global changes on local economies.
page 49
Finland
Broad Finnish approach is characterised by relevance and robustness of intervention
combined with flexibility which allows changes to be made in relation to economic
conditions.
- Government decided at beginning of 2009 to emphasise job retention and job
creation and ESF was redirected to projects helping this. Regional Offices of
Ministry of Employment and Economy were advised to favour calls emphasising
job retention and job creation both in relation to AP1 and AP2, the first time that
ESF measures had been so clearly aligned to government initiatives. Previously
ESF had been addressing structural challenges in Finland and the shortage of
labour principally due to the ageing workforce, high levels of retirement and a
contracting labour supply.
- Change of direction was possible because of flexibility in OP design, written in
broad terms with large funding categories.
- Change of direction was able to be implemented effectively because the
managing structure of the OP allowed for this.
- The OP design turned out to be robust, flexible and appropriate in the face of the
changing economy and labour market.
Good Quality Staffing, Systems and Management Arrangements

Although many country reports provided evidence of issues around the management
and administration of ESF more generally, where staffing and management
arrangements are good quality this greatly assisted in the effective implementation of
ESF A2E as the examples below illustrate.
Good Quality Staffing, Systems and Management Arrangements
Czech Republic
Competent teams within beneficiary organisations with responsibility for managing
services.
Clarity of communication, division of responsibilities and the setting of clear rules for
employees within beneficiary organisations.
Hungary
In terms of processes, the IB is developing personalised relationships between IB
officials responsible for particular calls and project leaders.
Italy
Activities delivered in smaller territories do better perhaps due to the greater ease of
project management and closer relationships between stakeholders and agencies.
Institutional learning helps improve intervention effectiveness, including making more
use of evaluation findings which seem to be more prevalent in small territories.
Netherlands
The Municipality of Emmen has created an ESF office. The office coordinates
applications and the implementation of projects and serves as a base of reference on
ESF for all municipalities in the Dutch province of Drenthe.
The Training and Development Fund of the Fire Service has created a project
organisation that coordinates ESF programmes within this sector. The organisation
focuses on the provision of information on the quality of administration.
Poland
Prior experience of project providers and beneficiary organisations is a key factor in
facilitating effective implementation.
page 50
Spain
A number of the bodies administering ESF have great expertise. The OP to fight
discrimination is unique and has been identified as best practice in the EU.
- A number of the IBs are national level NGOs with significant expertise working
with vulnerable groups.
- At project level there are regional and local NGOs, trade union and employer
organisations again with great expertise and experience.
United Kingdom
The approach to contract and performance management has a bearing on effective
performance and delivery in respect of ESF provision. Important factors include:
- Openness and honesty in relationships between contract managers, prime or
lead contractors and their subcontractors or delivery partners.
- The use of performance targets used at different levels within the ESF delivery
system although, in a minority of cases, subcontractors and delivery partners to
feel that the targets given to them by prime and lead contractors were unrealistic.
- The use of result-based systems of payments to providers was perceived
beneficial in supporting performance improvement and encouraging providers to
focus on the achievement of positive results for ESF participants. However,
result-based systems were also seen by some as leading to a focus on achieving
outputs at the expense of ensuring quality in provision or targeting of individuals
perceived as being easier to help and more likely generate job results.
Quality and Effectiveness of National Institutions

A number of country reports emphasised the importance of national institutions in
adding value to ESF A2E interventions. References made elsewhere to the key role of
the Public Employment Service in terms of finding work for recipients of ESF A2E, but
other agencies and organisations have a significant role to play.
Quality and Effectiveness of National Institutions

Czech Republic
The external operating environment is critical, including things such as national
legislative support, attitudes of public towards finding solutions to problems of target
groups, and the quality of existing A2E support services.
Ireland
Collaboration across different types of organisations drawing on individual strengths
(reach, referral, delivery of programmes, access to job placement and work
opportunities) evaluated positively.
Portugal
Capacity of PES to cope with increased unemployment and at the same time provide
tailored approaches is a key feature.
5.3 Good Practice in implementation

Most of the country reports present evidence on good practice in implementation. As
with much good practice reported, there always need to be major health warnings
around it, but these are the best assessments by the experts of what constitutes
effective implementation in their countries. Below we organise the discussion of good
practice under a number of headings.
page 51
Engaging Hard to Reach Priority Groups
A2E interventions such as vocational training, confidence building, etc. often struggle
to achieve significant results for recipients. There is often however a prior issue. This is
how to get the more disadvantaged groups involved in the A2E activity in the first
place. General good practice in this area tends to focus on working through
organisations, typically in the non-governmental sector, who already work with the
particular sub-section of the population. This could include for example housing
organisations responsible for housing in areas where there are high concentrations of
disadvantaged groups. In relation to ethnic minority groups the engagement problem
can be severe.
Engaging Hard to Reach Priority Groups
Belgium
Programmes in the Brussels region engaged a high proportion of participants of non-
Belgian ethnic origin but at the same time achieved very good gross and net
employment results. This is unusual as this target group has proven previously hard
to engage and assist into employment.
Denmark
E-learning to increase labour market competencies in remote areas has been
effective.
United Kingdom
Quality outreach was important to securing additionality - the recruitment of inactive
black and ethnic minority participants required the partnership of many agencies.
Customising the A2E Package of Interventions

A number of the experts commented on the importance of designing A2E interventions
around the specific and often complex needs of recipients, particularly those with the
most disadvantages in terms of aspiring to sustainable employment. This often
involves the provision of activities which help build confidence, raise self esteem and
enhance employability skills all things upon which subsequent activities can build.
Customising the A2E Package of Interventions
Austria
Employment Companies are mainly targeted at long term unemployed giving
opportunity to get back to realistic workplace settings. At the same time, the people
are supported to help raise self confidence and self esteem. Evaluation evidence
suggests that these:
- Help raise employability but also create additional jobs for people furthest from
the labour market through secondary labour market.
- Provide a safe and confidence building atmosphere in a work situation making it
easier for furthest from the labour market to re-integrate into labour market.
- Work best for low level activities such as home and office cleaning services,
facilities management, parts, textile recycling, etc.
- Work less well where higher qualifications, work experience, physical fitness and
soft skills are required.
Hungary
The experience is that for more effective complex assistance to respond to complex
needs of some A2E target groups, it will tend to be associated with a lengthier
process before results are achieved. This has implications for how OPs are designed,
monitored and evaluated.
page 52
Ireland
Labour Market Activation Fund (LMAF) emphasises the need to focus on the
development of softer skills particularly for the long term unemployed or those whose
employment background had been very specific. The good practice approach is for
core skills development to be linked into an intervention pathway leading to
employment.
Italy
Tools that seem to work well have:
- High level of personalisation.
- High level of flexibility.
Ad Personam Service (Trento) and individual voucher system (Lombardy) operate
voucher systems which appear to give recipient capacity to purchase directly from a
network of accredited providers. Payment for services made to providers directly by
MA. This forces the PES and other service providers to look at clients as customers
with individual needs.
Other examples of tailor made (customised) activities include internships, work
experience, incentives and research fellowships. The key ingredients here are:
- A written agreement.
- Promotion of learning experiences in and out of the place of work.
- Development of CSR practices in offer of suitable opportunities by employers.
Poland
Particularly for most disadvantaged groups, delivery of a complex support package
with the right provider is good practice. One example is of an intervention where the
project includes psychological and group support, vocational advice, job agency,
internships and cooperation with employers. A key feature is that the project provider
has significant experience with the client group, with a customised approach to each
participant as well as time committed to working with participants families. Project
process was well planned and monitored especially in relation to recruitment, and
20% found employment, from a particularly disadvantaged group.
Romania
Customisation is achieved in some activities by the development of a systematic
ladder of progression through various measures to assist individuals back into regular
employment on an incremental basis.
United Kingdom
Personalised training programmes, involving one on one support for participants as
well as flexible and open-ended delivery were highlighted as a particular success.
Interventions for Specific Groups

There are some indications of good practice which works well for specific target
groups, although there is a disappointing amount of evidence on this important issue.
The box below provides some illustrative material.
Interventions for Specific Groups

Women
Poland
In relation to equal opportunities for men and women, the Polish MA introduced
minimum standard from 1 April 2009 where project assessment involves 6 questions
on project implementation from a gender perspective to drive up the standards of
projects delivering ESF on the gender equality issue.
page 53
Portugal
Programme Bero de Emprego supports the costs of replacing a worker on maternity
leave with an unemployed woman, diminishing some of the resistance to hiring
women. Half the substitute recruits end up being hired.
Young People
Austria
Due to the recession, greater emphasis has been put on getting young people into
employment. Apprenticeship training was introduced by the PES to make up for lack
of apprenticeships in companies, backed up by a government pledge that all young
people in Austria would be offered an apprenticeship. This pledge has been realised.
Youth employment measures help companies pre-select for apprenticeships, but
there is the danger that the apprenticeship measures themselves displace
conventional apprenticeship as companies get the taxpayer to pick up the cost.
Interventions significantly increased the number of young people with qualifications.
Young people with other issues received support through ESF co-funded measures
which helped build their self esteem and self confidence.
Denmark
There has been a focus on targeting youth education and the national target of 95 per
cent of each cohort completing youth education. This has been supported by ESF
funded activities such as Hold Fast which focused on using youth councillors as
mentors as well as other activities to avoid young people dropping out of education.
Linking Individual Need and Labour Market Opportunity

Effective implementation of A2E interventions requires a focused approach to two
different sets of customers the individual we are trying to progress to sustainable
employment and the employer who will be critically important in realising this
objective.
Linking Individual Need and Labour Market Opportunity
France
One of the ESF funded projects involves one to one work with job coaches which
facilitates the delivery of a customised service and a link to potential employers and
77% of recipients secured a permanent job. The key point is that job coaches work
both with the individual unemployed person and potential employer to make an
effective match.
Ireland
An example of targeting individual need and market opportunities is an assessment
and guidance programme for 1,000 unemployed people to provide individualised and
person centred education and training needs as well as matching jobseeker skills,
experience and interests with labour market opportunities.
Another example concerns the added benefits from work placements in training
programmes where there is clarity about goals, timelines and deliverables. This helps
both participants and employers understand what is the desired result and therefore
makes it more likely that the result will be achieved.
Both LMAF 2010 and review of labour market programmes emphasised the need of
education and training to give jobseekers skills to meet labour market and specific
industry needs, including opportunities emerging in new sectors prioritised in national
policy (e.g. renewables). Additionally, direct links to employment opportunities
through structured work experience and placement programmes are key success
factors.
page 54
Some project providers have engaged directly with employers to establish their skill
needs, to create or modify existing courses to meet these and then bring participants
onto those courses with a view to raising their employability.
Value Added by Partnership Working

There is a growing recognition of the importance of partnership working, particularly in
relation to the A2E agenda. For the more disadvantaged recipients of A2E
interventions they may well need the support of a range of different organisations
drawn from, for example, health, training and job finding services. Some examples of
good practice in partnership working are set out below.
Value Added by Partnership Working
Ireland
The range of partnerships and collaborative approaches involved in the
implementation of ESF A2E in Ireland were highlighted as adding significant value.
LMAF 2010 argues the value of the holistic approach to A2E. ...the mix of core and
specific industry and occupational focussed skills, the inclusion of a work placement
aspect, the provision of wider forms of support addressed to needs of individual
participants and the focus on progression rather than education and training courses
per se were identified as key characteristic of success.
United Kingdom
In the West Midlands the adoption of a consortium and network covering the whole
region was seen to have worked well allowing providers to learn from each other and
test fresh ideas and to develop a much keener understanding of the issues that each
of them are dealing with.
In Scotland, the Community Planning Partnership model, which to some extent
anticipates the Integrated Territorial Investment and Community Development
concepts advanced in the draft Regulations for 2014-20, was perceived to have
improved the efficiency of local employment services through improved coordination
and reduced duplication. Strategic commissioning was believed to represent better
value for money and to result in reduced administration compared with partnership
developed strategies and competitive bidding.
page 55
6 Community Added Value
Overall Assessment of Community Added Value

Community Added Value (CAV) is a measure of the difference made by the fact that
ESF A2E is funding and programming interventions compared to purely Member State
funding and programming.
The methodological note prepared by the Evaluation and Impact Assessment Unit in
October 20087 identifies 4 key dimensions of CAV.
Volume Effects. ESF action adds to existing action, either by supporting
national action in general (mirroring) or specific areas of national policy
(boosting).
Scope Effects. ESF action broadens existing action by supporting groups or
policy areas that would not otherwise receive support.
Role Effects. ESF action supports local/regional innovations that are taken up
at national level or national innovative actions that are then mainstreamed.
Process Effects. ESF action influences Member States administrations and
organisations involved in the programmes.
A key implication of this approach to defining CAV is that the appropriate measurement
of CAV will vary from one OP to another, dependent on how ESF resources have been
deployed.
In the event, the treatment of CAV in the country reports is extremely patchy.
Some make little or no reference to it at all (CY, FR, GR, NL, SE).
Others say there are no studies to base their assessment on (LU, RO).
Some hazard a guess as to the likely CAV resulting from ESF A2E interventions
(IT, LV).
The remainder provide a more structured and evidenced assessment.
In a small number of country reports (DE, IE) the treatment of CAV is nicely structured
around the EC definition with assessments of all four key aspects of CAV.
7 A Framework to Describe the Community Added Value of the ESF (2008) Methodological Note: EC
Development, Social Affairs and Equal Opportunities DG, Evaluation and Impact Assessment Unit.
page 56
CAV for 4 Key Dimensions
Germany
The Federal states have a substantial share of EU funding, a level of resource that
they would otherwise struggle to assemble. These can be deployed to meet the
needs of their regions.
European policy-orientation directs money into areas that would be served to a lesser
extent by national and regional OPs, e.g. in relation to migrants and asylum seekers.
ESF allows for more innovative and intensive support for groups such as those with
multiple disadvantages in the labour market.
EUs governance principles (including indicator based monitoring and controlling,
regular reporting and evaluation, networking and mainstreaming, and broader
stakeholder involvement) impact upon national administrations.
Ireland
ESF makes a substantial contribution to the resource available for A2E which has a
significant impact given the difficulties of the Irish Exchequer.
ESF helps to ensure that there remains a strong focus on gender, wider equality and
social inclusion in relation to A2E activities.
The Labour Market Activation Fund was also able to draw on experience of EQUAL
with innovation and new models of delivery.
ESF co-financing helps to ensure a level of planning, objectivity in target setting,
management, monitoring and reporting discipline that is unlikely to exist in its
absence.
In virtually all cases where an assessment is made CAV is deemed to be positive. On

the assumption that the relatively shallow treatment of CAV in many of the country
reports is a function of the level of treatment of CAV within individual evaluations, the
Commission is faced with a difficult problem in finding the evidence of CAV, and it
should require MS to strengthen this analysis.
Volume Effects
Volume effects emerge in the country reports as the most common form of CAV.
They are particularly significant for newer members of the EU that have more
limited public employment services and ALMPs.
They are also important however for older members of the EU and it is these
countries that secure the bulk of ESF A2E funding.
The importance of volume effects has clearly been heightened by the effect of
the economic crisis which has both brought forward a massive increase in
people requiring A2E assistance as well as a decline in the capacity of public
finances within Member States to resource the necessary interventions.
Some illustrations from some Member States are provided below.
page 57
CAV Volume Effects
Austria
ESF co-funding was seen by TEPs as a means of doubling their budgets for projects
for people furthest from the labour market.
Czech Republic
ESF significantly increases the total amount of resource allocated to A2E in the
range 75-100% of resources for activities delivered.
In the absence of these resources, it is estimated that 88% of recipients of PA2 and
90% of recipients of PA3 would suffer a deterioration in their situation.
Estonia
The increased volumes of A2E activities now being delivered would not be possible
without ESF support.
The sustainability of activities introduced and the capacity to introduce new activities
would not be impossible without ESF support.
Both of the above are intensified by the economic crisis, and the capacity to mitigate
the worst of the crisis would not have been possible without ESF support.
Portugal
The AIRs reveal widespread perception of considerable CAV from ESF. Levels of
activities, outputs and results would not be achievable with domestic resources alone
in relation to A2E and other policy fields.
Slovak Republic
ESF is critical for the delivery of A2E interventions, providing 54% of the funding
required to achieve the ALMP objectives of the Member State.
Slovenia
65% of the Slovene Active Employment Policy is funded by ESF and means that
existing interventions are much better resourced and assist many more final
recipients.
United Kingdom
Evaluation evidence suggests considerable added value in terms of increased
volumes of unemployed recipients.
Scope Effects
It is clear that scope effects are a very significant aspect of ESF A2E CAV, as the box
below illustrates. There are two elements here that stand out:
Support for specific groups (such as immigrants) that would not otherwise have
received as much support, or indeed any, in terms of A2E interventions.
Support for specific types of intervention, typically most costly and labour
intensive ones, which would be less likely to be available to A2E target groups.
page 58
CAV Scope Effects
Belgium
The ratio of ESF final recipients to potential recipients for 2010 amounted to 23% of
the unemployed, but 51% of disabled unemployed people.
Czech Republic
ESF projects both made it possible to work with larger number of individuals in target
groups, but also with completely new target groups.
Estonia
The at risk and other target groups associated with ESF funding support includes
some groups who would have received limited or no A2E services from the Member
State.
Lithuania
Particularly for clients with the potential to impose significant costs on public services
(e.g. ex-offenders), there is some evidence that interventions both created better
employment prospects but also helped with prevention (of crime in this instance).
Similar findings emerged on clients with addictions.
Malta
A number of the key interventions were completely new and additional to what would
otherwise have occurred, as they were not previously included in local plans. This
refers particularly to the programmes for women and for disadvantaged and
vulnerable groups.
Poland
CAV is connected with ESFs stronger focus on interventions aimed at the activation
of young unemployed persons in the labour market, and the setting up of indicators
and measurement processes. For this group policy on A2E has changed from passive
to active, and ESF has taken the lead and set standards on how to do it.
ESF funding for the central OP supports the development of interventions for groups
covered in Polish law which have specific features increasing their disadvantage e.g.
ex-offenders and members of Roma community. ESF supports tailored interventions
to meet the needs of these groups.
The Polish labour fund works with registered unemployed individuals, whereas ESF
interventions are addressed at workless people more generally including the inactive.
Slovenia
As a result of ESF a number of additional programmes have been added to the
Slovene Active Employment Policy, including on-the-job training for graduates,
subsidies for employment in the not-for-profit sector and support for specific
geographic areas.
Spain
ESF A2E support has led to a much more significant efforts in relation to the social
and labour market integration of immigrant groups.
United Kingdom
As a result of ESF support a range of key interventions are delivered, or delivered to
a much higher quality, than would have otherwise been the case. This includes
outreach activity and Intermediate Labour Markets, generally targeted at the more
disadvantaged groups of the population.
page 59
Role Effects
Role effects appear less frequently in the assessments of CAV, possibly due to the
substantial investment already made by the EC in innovation through programmes
such as EQUAL.
CAV Role Effects
Austria
As the projects to receive ESF support had to bring forward new approaches to the
integration of the target groups role effects in terms of innovation were realised.
Estonia
The ability to develop innovative approaches (e.g. different training courses for
different target groups, measures to deal with disrupted studies, etc.) was stimulated
as a result of ESF support.
Finland
ESF funding is significant in triggering national innovations in enhancing employment
and skills. An example is the intervention introduced to promote job to job transitions
in redundancy situations.
A strategic evaluation found that 70% of activities would not have been launched
without ESF funding, and that it was particularly important in terms of promoting skills
and employability of jobseekers and finding new ways of delivering these results.
Process Effects
Process effects also appear less frequently as CAV in the country reports than might
have been anticipated, although it is not surprising that the three examples reported
below all refer to countries joining the EU in more recent times.
CAV Process Effects
Hungary
The volumes of A2E interventions and recipients were significantly impacted by ESF,
but ESF has also contributed to experimentation and to reforming the practices of the
PES. There are considerable costs to the PES in making the required changes which
would not have been implemented without ESF support.
Malta
Accessing ESF required wide consultation with stakeholders and partners to
understand what kinds of interventions were required, and to customise these to local
circumstances.
The OP has made significant demands on institutions and systems, including
information and control systems. This applies to both public and private agencies.
Poland
The activities of the Human Capital OP contribute significantly to an increase in
common activities and approaches across public institutions. This cooperation is likely
to result in the identification and implementation of optimal solutions in relation to A2E
and Social Inclusion policies, as well as making for more effective operational working
across public institutions.
page 60
6.1 Variation in CAV by priority group and type of intervention

Due to the limited treatment of CAV in the evaluation material available to the experts,
there is virtually no scope to consider whether different types of interventions of
assistance for specific priority groups generates greater CAV per mn deployed.
The UK report is interesting on this issue as it does report that ESF funded A2E
produced better employment results compared to mainstream UK provision for the
unemployed, based on a valuable bench-marking exercise. Nevertheless the report
also argues that substantial ESF support has been provided to a large number of
unemployed job seekers relatively close to the labour market with limited employment
durations and that many of these may have found work without ESF A2E support. It
is this kind of intelligence which is generally lacking.
page 61
7 Evaluation data, approaches and methodologies
7.1 Need for good quality evaluation evidence

One of the key research questions is whether the data available and the approaches
and methodologies adopted allow for robust evaluation of the results flowing from ESF-
funded A2E key areas of intervention. In this chapter we will examine these issues.
Before doing so, it is worth considering the information that evaluations could usefully
provide as this will give us a framework against which to examine the evaluation
data, approaches and methodologies used.
At the most basic level, the evaluations should identify the results of the
programme (i.e. whether recipients have moved into sustainable employment)
and the cost of delivering these results. Ideally information on results should be
available for different intervention types and target groups so the evaluation can
assess both what works and for whom.
Building on this, evaluations that are able to benchmark ESF-funded A2E results
against those achieved by national programmes or the results for the
unemployment register caseload are useful in helping identify the added value of
ESF to a Member State.
Counter-factual evaluations seek to establish how the results achieved compare
to what would have happened if ESF funding for A2E had not been in place. The
most common method of counter-factual evaluation is to compare recipients of
(in this case) employability services through ESF A2E with individuals with very
similar characteristics (referred to as the control group) that did not access the
ESF A2E interventions.
As an alternative to control groups, econometric modelling can be used. This
would allow the likely structure and distribution of (say) long term unemployment
within a Member State with and without the resource committed through ESF
A2E to be modelled.
This section of the report reviews the data and approaches to evaluation available to
the country experts to help prepare their country reports.
There has been some updating of some of the key tables from the first Synthesis
Report which focused strongly on evaluation approaches and methods. The
updated tables are discussed briefly in the Annex to this chapter, as this
assessment of the evaluation materials is not restricted to evaluations
contributing to ESF-funded A2E.
A review of the data sources, quality and reliability is presented based on the
material in the country reports focussing on ESF-funded A2E.
An analysis of evaluation approaches and methodologies is also carried out
based on the country reports.
Some implications are noted flowing from the review of data and evaluation
methods.
7.2 Evaluation data sources, quality and reliability

Country experts were asked to review all relevant evaluation evidence. However, most
experts went beyond the evaluation evidence to include AIRs and relevant research
studies to develop a more complete picture of the impact ESF-funded A2E efforts were
having. Table 7 summarises the sources of data underpinning the documents reviewed
by country experts in compiling their country reports. The key points to note are:
page 62
Output data is the most common type of data utilised with 133 evaluations
making use of this type of data. DE, ES, PL and UK in particular appear to make
extensive use of this data although this in part reflects the large number of
evaluations undertaken in these countries.
Other forms of secondary data are also widely used, with 113 evaluations
making use of results data based on monitoring, 112 making use of financial
data, and 95 making use of final recipient contact data.
Less common is primary research, with only 82 evaluations using results data
based on research.
All countries (for which we have information) make use of more than one type of
data. Many draw on all four types of data.
In terms of the accessibility, quality and reliability of data:
In most instances data were relatively accessible to experts, and hence to other
interested parties as required.
Data reliability was also not a great concern although system changes over the
life of the 2007-2013 programme meant that data currently available were often
not comparable with earlier data.
In relation to data based on research, some issues that were identified included:
The sample selected with some seeking statistical representation and others
aiming to capture a range of experiences (for example, through case studies).
The sample size with cost issues at play here. In particular, whilst overall
sample sizes may be statistically robust, they may not be sufficient to enable
analysis of specific target groups (e.g. ethnic minorities, individuals with a
disability, young people) or interventions.
Attrition in cohort surveys with diminishing participation in successive waves.
This requires reweighting of the survey results to bring them in line with the
original sample specification and to facilitate meaningful comparison between
survey waves.
However, it is clear from many of the country reports that experts struggled to find
data that related specifically to ESF A2E types of operations focussed on the types
of final recipients prioritised under A2E. This applied to expenditure, number of final
recipients and to employment results. On employment results, there was virtually no
information on the sustainability of employment secured following an ESF A2E
operation. Additionally, for employment results many country experts were dependent
on monitoring data provided by beneficiary organisations raising issues about the
reliability of the information.
Returning to the hierarchy of information that the evaluations could potentially provide:
Most weight, simply due to availability, was given to monitoring data of one kind
or another.
- The data available varied across Member States and evaluations and
therefore the capacity of evaluations to meet the most basic requirement (i.e.
to be able to identify the results and the cost of delivering these) was at
times, limited.
- In addition, most of the monitoring data available was provided by beneficiary
organisations with no independent verification. This limits the confidence
page 63
that can be placed in the results reported. Further problems result when
beneficiary and intermediate organisations maintain data on different systems
and/or interpret indicators in an inconsistent manner.
- The one key exception to this is where Member States have in place central
databases (often managed by PES or tax authorities) that contain information
on all adults of working age and evaluations have been able to draw on this.
As well as being more robust than data provided by beneficiary
organisations, this can allow some benchmarking to be undertaken.
However, this only appears to have been undertaken in a small number of
cases.
Few drew on primary research and only a sub-set of these will have used
counter-factual (control group or econometric modelling) techniques (the others
may, for example, simply have surveyed recipients about their experiences of
intervention). This means very little information is available on the additionality of
ESF-funded A2E interventions.
page 64
Table 7. Evaluation data sources
AT BE BG1 CY2 CZ1 DE DK EE ES FI1 FR GR1 HU1 IE1 IT1 LT1 LU LV2 MT3 NL PL PT RO1 SE SI SK UK Total
Final 7 3 2 - 3 13 5 2 5 1 4 4 6 2 14 3 0 - 0 0 11 0 2 0 0 0 8 95
recipient
contact
data
Financial 3 4 2 - 2 21 7 1 9 3 3 4 2 3 14 3 1 - 4 1 5 6 3 1 1 1 8 112
data
Output 5 4 3 - 2 25 3 3 9 1 3 4 7 4 16 3 0 - 4 1 13 5 3 1 1 1 10 131
data
Results 5 7 2 - 3 15 3 3 9 2 3 4 6 4 7 3 1 - 4 1 14 3 3 1 0 1 9 113
data
based on
monitoring
Results 6 5 2 - 1 18 0 2 0 1 1 0 5 1 8 1 0 - 0 0 15 0 2 0 0 0 14 82
data
based on
research
Notes:
1. Bulgaria, Czech Republic, Finland, Greece, Hungary, Ireland, Italy, Lithuania and Romania - Calculated by TERU based on individual evaluation information provided
in Annex.
2. Latvia and Cyprus information provided in Annex confusing so not included here.
3. Malta Table provided is for All AIRs reporting data. Country experts accessed AIRs for 2007-2011, so attached figure of 4 to each
page 65
Although a number of issues with data sources quality and reliability were identified in
some of the country reports, there are also some good examples to draw upon which
are captured in the box below.
Good Practice in Data Availability, Quality and Reliability
Austria
Austrian monitoring data tend to be very accurate and evaluators generally have
access to a data warehouse (DWH) which gives an excellent breakdown of
participant data. This is run by the Austrian PES and Ministry for Labour, Social
Affairs and Consumer Protection. This provides anonymous person and company-
related information e.g. employment and unemployment episodes, periods out of
employment, etc. Research often is at the individual participant level. Evaluations can
draw on focus groups as well as surveys of participants.
In general, monitoring data are used to provide information on the absorption of funds
and overall progress. Research data are then used to explain the results.
Belgium
Evaluators in Belgium are able to draw on 3 different methods (beyond intervention
monitoring data) to collect information on results of ESF-funded A2E interventions.
- Crossroads Bank for Social Security which can identify whether an individual in a
given quarter of the year is on benefit, in paid employment, etc.
- PES Database which can show on a monthly basis whether an individual is still
unemployed or has left unemployment.
- Surveys of individual recipients.
Germany
In relation to general statistics, accessibility, quality and reliability are good. For
example, the PES has its own research institute and has considerable and lengthy
experience in providing, generating and analysing appropriate data. These data can
be readily disaggregated to quite small geographies.
In terms of programme monitoring data, Germany has a well established and
reasonably common approach built up from the individual projects, including
enterprises where these are the target recipients.
There is also a system which is simpler involving less documentation to be completed
by participants. This is for:
Interventions with low-level intensity.
Interventions addressing systems, rather than individuals or
enterprises/organisations.
Poland
There are two systems of data collection to help monitor the effectiveness of ESF in
Poland.
- National Information System (SIMIK) which collects data on OPs, payment
applications, non-compliance, etc. The system charts expenditure and
expenditure forecasts.
- The monitoring sub-system of ESF 2007 (PEFS) gathers data for monitoring the
achievement of goals through ESF interventions including information about final
recipients.
13 evaluations used participant surveys which are easy to conduct because PEFS
has contact details for final recipients. These surveys help measure intervention
effectiveness.
page 66
7.3 Evaluation approaches and methodology

Table 8 summarises the approaches and methodologies used in the evaluations
included in this exercise. As you would expect many evaluations use more than one
approach/methodology (for example, looking at management and implementation
processes, analysing monitoring data and undertaking primary research) and therefore
a single evaluation can appear in more than one column. In addition, whilst most
evaluations are allocated to a single row, where the evaluation design and scope
requires it, some evaluations appear in more than one row. For this reason, the column
totals should be treated with caution as there is a degree of double-counting within this
table. The column totals should be considered as providing simply as indicator of the
relative usage of different evaluation approaches/methods.
Key points emerging from this analysis include:
The use of monitoring data is widespread. This is the most common
methodology overall and was the most commonly used methodology for
evaluations that specifically focused on A2E interventions and target groups
(used in 60 evaluations of this type), evaluations that specifically focused on a
subset of A2E interventions and target groups (43) and evaluations that focused
on a target group, a subset of which includes A2E final recipients (29). Every
Member State that reported on their evaluation approaches and methodologies
(except LV and SI) had used monitoring data in at least one evaluation.
Similarly, analysis of management and implementation processes was
extremely common across all evaluation approaches. Its use was particularly
common in evaluations that specifically focused on A2E interventions and target
groups (used in 50 evaluations of this type) and evaluations which focus on
intervention type where a subset of the target group are A2E final recipients
(48). Every Member State that reported on their evaluation approaches and
methodologies (except NL and SI) had analysed management and
implementation processes in at least one evaluation.
Evaluations drawing on primary research (evaluation of results based on final
recipient case study/focus group and evaluation of results based upon final
recipient research) were much less common.
Impact evaluations were rare and where they did exist were more likely to be
theory-based than counter-factual or other impact evaluations.
There is no clear pattern across the different evaluation approaches (e.g.
evaluations specifically focusing on A2E interventions and target groups, etc.) in
terms of whether impact evaluation has been used or in terms of the choice of
impact evaluation method (theory-based, counter-factual or other).
There is also no clear pattern as to which Member States have undertaken
impact evaluations with both EU-12 and long-established Member States
represented, as are both those with multiple OPs operating at national and
regional levels (e.g. DE and ES) and those with just a single OP.
Some country experts challenged the rigour of the counter-factual evaluations
that had been undertaken, particularly in relation to control group selection.
Again, the evidence points to:
page 67
The focus being on approaches/methodologies based on monitoring data of one
kind or another which at best will be able to identify what the results have been
and the cost of achieving these. However, in practice, not all evaluations were
able to meet this most basic information requirement.
The approaches/methodologies used suggest limited efforts to benchmark
against national programmes, other interventions or unemployment benefit off-
flows.
Similarly, there has been limited use of counter-factual evaluation to establish
the additionality associated with ESF A2E expenditure. The limited number of
impact evaluations reflects the difficulties involved in designing and delivering
these, including accessing data particularly micro data; identifying, recruiting
and sustaining control groups including data protection issues; and the stage
in the programming period with most Member States focussing on monitoring
implementation rather than evaluating results.
page 68
Table 8. Evaluation approaches and methodologies

Evaluation of Evaluation of Evaluation of Evaluation of Theory Based Counter-factual Other Impact
Management and Results Results Results Impact Evaluation Impact Evaluation Evaluation
Implementation Using Monitoring based on final based upon
Processes Data recipient Case sample final
Study / Focus recipient research
Group
Evaluation (BE 4) (DE13 ) (BE 8) (DE 13) (ES (BE 1) (DE 3) (ES (BE 1) (DE 7) (ES (DE 3) (ES 2) (GR (BE 2) (HU 2) (IE (DE 2) (IE 1) (LT
specifically (ES 10) (FR 3) (GR 12) (FR 3) (GR 6) 7) (FR 2) (IE 1) 1) (HU 1) (IE 2) (IT 5) (PL 1) (UK 6) 2) (LV 1) (UK 2) 1) (UK 1)
focusing on A2E 5) (IE 1) (LT 1) (LV (HU 3) (IE 2) (IT 2) (LT 1) (UK 2) 1) (LT 1) (LV 1) (Cell total 17) (Cell total 9) (Cell total 5)
interventions and 1) (MT 4) (UK 8) (LT 1) (MT 4) (PL 1) (Cell total 17) (PL 1) (UK 7)
target groups (Cell total 50) (SE 1) (SK 1) (UK (Cell total 23)
3) (Cell total 60)
Evaluation (AT 2) (DE 6) (ES (AT 1) (DE 6) (ES (AT 3) (DE 6) (DK (AT 1) (DE 8) (DK (DE 3) (GR 5) (HU (IT 1) (PT 1) (Cell (DE 6) (HU 1) (PL
specifically 4) (GR 5) (HU 1) 4) (GR 6) (HU 3) (IE 2) (ES 2) (HU 2) 2) (HU 1) (IT 4) (LT 3) (LT 1) (PL 7) total 2) 1) (Cell total 8)
focusing on subset (LT 1) (MT 4) (PL 1) (IT 1) (LT 1) (MT (NL 1) (PL 5) 1) (PL 8) (UK 1) (UK 1) (Cell total
of A2E 2) (PT 5) (UK 2) 4) (NL 1) (PL 9) (PT (Cell total 21) (Cell total 26) 20)
interventions and (Cell total 32) 5) (UK 1)
target groups (Cell total 43)
Evaluation focused (AT 2) (ES 4) (GR (AT 2) (BE 1) (DK (AT 2) (BE 1) (ES (AT 2) (ES 1) (IT 2) (AT 2) (GR 6) (HU (AT 2) (LT 1) (Cell (GR 5)
on target group, a 5) (HU 1) (LT 1) 7) (ES 5) (GR 6) 4) (HU 1) (LT 1) (LT 1) (LV 1) (PL 1) (PL 3) (UK 1) total 3) (Cell total 5)
subset of which (PL 1) (UK 4) (HU 1) (IT 2) (LT 1) (PL 2) (UK 1) 3) (UK 2) (Cell total 13)
includes A2E final (Cell total 18) (PL 3) (UK 1) (Cell total 12) (Cell total 12)
recipients. (Cell total 29)
Evaluation which (AT 2) (BE 16) (DK (AT 1) (BE 9) (ES 3) (AT 1) (ES 1) (FR (AT 2) (IT 3) (PL 2) (AT 1) (ES 2) (GR (Cell total 0) (Cell total 0)
focuses on 7) (ES 2) (GR 5) (FR 1) (GR 6) (IT 5) 1) (IT 1) (PL 2) (UK 2) 5) (PL 3) (UK 2)
intervention type (IT 8) (MT 4) (PL 2) (MT 4) (PL 2) (UK (UK 1) (Cell total (Cell total 9) (Cell total 13)
where subset of (UK 2) (Cell total 2) (Cell total 33) 7)
the target group 48)
are A2E final
recipients.
Total 148 165 57 70 63 14 18
page 69
Table 8 gives a good overview of the different approaches to evaluation. However, it is
essential to record a concern raised in a number of country reports. This relates to the
fact that there is no longer a requirement from the EC for MAs to carry out Mid-Term
Evaluations. Some country experts felt that this meant there is a risk that robust
quantitative information on the implementation of the 2007-2013 Programme may not
be available until 2013. Mid-Term Evaluations, as well as providing good quality
quantitative and qualitative data, allow an assessment to be made of any critical
changes in economic and labour market contexts, national budgetary positions, etc. In
addition, they can review the effectiveness of implementation and the structures for
managing this. All of this feedback allows MAs to consider carefully and on the basis of
robust evidence whether, to what extent and how to change their OPs. The AIR
reporting process does not generate this quality and robustness of feedback, but
perhaps the AIR process could be enhanced to provide some of the benefits that flow
from Mid-Term Evaluations.
Although there are some issues with the evaluation approaches and methods used to
try to identify the activities, outputs and results flowing from ESF A2E, there are also
good practice approaches in a number of Member States, captured in the box below.
page 70
Good Practice in Evaluation Approaches and Methodologies
Hungary
Recent evaluations have begun to take a more scientific approach and utilise more
demanding methodologies than those undertaken earlier in the OP.
- The dominant evaluation methodology is quantitative analysis based on
monitoring data although there is a slow move away from this orientation.
- Most of evaluations involve specifically focussing on A2E interventions and target
groups.
- There are some theory based and counter-factual impact evaluations, and it is
also quite common to find evaluation of results based on case studies or focus
groups, or surveys of final recipients.
However, impact evaluations are relatively rare because necessary data is missing or
hard to access, problems of identifying an appropriate control group and identifying
causality.
Latvia
Support for the implementation of regional action plans for promotion of employment
(WWS) is the flagship response of Latvian employment policy to the growth of
unemployment in recession. The principal aim of the programme was income support
for low income unemployed to bridge the gap of low or no income between jobs. The
WWS evaluation is an impact evaluation carried out on a counter-factual basis using
control groups. It also used content analysis, analysis of normative documents and
policy planning documents, a survey of households, a survey of municipalities,
interviews with representatives of municipalities, a survey of experts and analyses of
secondary statistical data.
Poland
The Polish approach to evaluating ESF A2E is multi-faceted.
- Most evaluations focus specifically on sub-set of A2E interventions and target
groups.
- To measure results, most evaluations use monitoring data, final recipient case
studies or focus groups and research or survey data with final recipients.
- Theory-based impact evaluations are common, but no counter-factual impact
evaluations are available.
- Most evaluations use quantitative, qualitative and other methods to facilitate
triangulation.
7.4 A2E evaluation requirements and deliverability

As discussed above, ESF-funded A2E interventions generally have a specific result in
mind which is a sustainable employment entry. As such, the absolutely essential
evaluation requirement is to investigate in a robust way the extent to which this is
achieved. The evidence presented earlier in this chapter suggests that many Member
States do not currently have the systems or processes in place to gather this most
basic of information. A key issue here is that A2E is often delivered as part of a wider
intervention and it is not possible to separate out the A2E results. The large number of
evaluations noted in Table 8 that focused on a target group, a subset of which includes
A2E final recipients, and on intervention types where a subset of the target group are
A2E final recipients, is evidence of this key challenge.
Given that many evaluations are unable to present this basic information, it is perhaps
unsurprising that only a small number of Member States (AT, BE, HU, IE, IT, LT, LV,
PT and the UK) have used more sophisticated counter-factual techniques to establish
the impact of their ESF A2E spend. As discussed in the box below, counter-factual
evaluations can be of significant value but challenges in their implementation can
page 71
sometimes make it difficult for ESF-funded A2E interventions to be effectively
evaluated in this way.
Undertaking Counter-factual Evaluation

Counter-factual evaluations help establish whether an intervention makes a difference
(EVALSED)8. They build on other approaches and methodologies (such as analysis of
monitoring data, surveys of participants, etc.) that tell us what the results have been by
assessing whether or not these effects are caused by the interventions. For policy makers and
funders, it is the opportunity that counter-factual evaluations provide to establish the causal
link between interventions and results that is vital in helping them learn more about what
works in tackling the policy priority (in this case access to employment) and enables them
allocate their available funds effectively (Martini, 2009)9
As it is not possible to know what would have happened in the absence of the intervention (for
example, would a final recipient have moved into employment or stayed on the unemployment
register), the counterfactual is most commonly established using a control group of individuals
with similar characteristics to the intervention group. The control group approach to counter-
factual evaluations is widely practised across a range of disciplines and interventions. There
are, however, a number of challenges in undertaking effective counter-factual evaluation of
ESF-funded A2E interventions including:
Identifying, recruiting and sustaining matched samples (e.g. individuals with very
similar characteristics in terms of both the barriers to accessing employment they face
and the factors influencing these).
Identifying the results of the intervention. Control groups work best when the
interventions being evaluated are simple and causality is linear (White, 2010) 10.
Whilst A2E meets one criteria of simplicity in that the outcome can be clearly defined
(sustainable employment), this outcome can be the result of a number of
interventions (or in some cases, other factors such as a change in family
circumstances) and the interaction between these interventions/factors. This means
evaluators must take care in designing their counter-factual evaluation methods to
address this complexity.
Ensuring stability for the duration of the evaluation. A2E interventions learn as they
go, adapting their response in light of what works. In addition, it can be hard to
sustain the same conditions for the control group over time (for example, a new
player may start offering A2E services in the intervention area and some of the
control group members may tap into this).
Assessing the wider applicability results. A counter-factual evaluation provides us with
evidence on whether a specific intervention has had an impact in a particular set of
circumstances but this does not mean this has wider replicability (Stryczynski, 2009)11
Addressing ethical concerns whether it can ever be justified to deny an individual A2E
support in order to facilitate comparison (Reichardt, 2011)12.
8 EVALSED http://ec.europa.eu/regional_policy/sources/docgener/evaluation/evalsed/index_en.htm
9 Martini, A. (2009). Counterfactual Impact Evaluation: What It Can (and Cannot) Do for Cohesion Policy.
th
Paper to 6 European Conference on Evaluation of Cohesion Policy.
10 White, H. (2010). A Contribution to Current Debates in Impact Evaluation, Evaluation, Vol. 16, No. 2,
pp153-164. Sage Publications on behalf of The Tavistock Institute.

11 Stryczynski, K. (2009). Rigorous Impact Evaluation. The Magic Bullet for Evaluation of Cohesion Policy?
European Commission.
12 Reichardt, C. (2011). Evaluating Methods for Estimating Program Effects, American Journal of
Evaluation, Vol. 32, No. 2, pp246-272. Sage Publications on behalf of American Evaluation Association.
page 72
An alternative to constructing a control group is to undertake econometric modelling. Models

can include a number of different scenarios, the most common of which is intervention
on/intervention off. This allows the calculation of net effects. Delivering effective econometric
modelling to measure impact requires:
A good understanding of the causal relationships between factors. This can be
challenging due to the complexity of A2E efforts, with multiple interventions often
playing a role in generating the desired result (sustainable job entries).
Consideration of whether these causal relationships will hold over time. This is of
particular importance at the current time as the A2E relationships understood for the
period of economic growth up until 2007 may not continue to hold during the
economic crisis and its aftermath.
Given the vital importance of counter-factual evaluations in understanding the effects of A2E
interventions, finding ways to work around the issues raised above is essential. For example,
recent work by Crpon et al. (2009)13 has looked at addressing the challenges of programme
timing to ensure counter-factual evaluations of active labour market policies can be effectively
undertaken.
7.5 Some implications

A number of key points emerge from the analysis of data sources and evaluation
approaches.
The position varied greatly across Member States in terms of the opportunity to
evaluate specifically A2E interventions delivered to A2E target groups. Some
were able to do this quite precisely and others only in a very approximate
fashion.
The key message from this exercise is that it has not been possible to access
even the most basic of information (on the results of ESF A2E expenditure) in all
cases. If A2E is an important policy field, at the outset systems set up by
national MAs should have been tested to assess whether they could extract data
allowing A2E interventions for A2E target groups to be evaluated robustly.
Only once this condition has been met, can more robust methods
benchmarking results against national programmes or establishing control
groups be pursued. As these approaches can provide significant insights into the
added value of ESF, it is important that the systems are in place to support
these.
The variation across Member States in access to data linking interventions and
recipients within A2E mean that any comparisons made across Member States
in terms of results tend to be relatively meaningless.
13 Crpon, B., Farracci, M., Joliret, G., and van den Berg, G. (2009). Active Labour Market Policy Effects in a
Dynamic Setting. Institute for Labour Market Evaluation (IFAU) Working Paper 2009:1.
page 73
Annex to Section 7: Analysis of evaluations based on Inventory
Overview
On the basis of updated inventories as of 19 July 2012 the broad position is that:
107 evaluation plans have been prepared.
675 evaluations have either been completed, are ongoing or are still at the
planning stage.
Of the evaluations:
- 413 are operational evaluations.
- 206 are strategic evaluations.
- 287 are exclusively process evaluations.
- 103 are exclusively impact evaluations.
The relatively limited number of impact evaluations helps explain the difficulty in finding
comprehensive data on results encountered in the country reports focussing on ESF-
funded A2E.
page 74
Table A6.1 shows how finalised and on-going evaluation approaches vary across
Member States.
More than half of the impact evaluations were carried out in 2 Member States
(HU and PL). This underlines the fact that impact evaluations are relatively
uncommon across most Member States.
Even in relation to process evaluations there is a high degree of concentration
with 2 countries (PL and IT) accounting for more than 50%.
There is a major job to be done to generalise both process and impact evaluations
across all Member States in sufficient quantities to guide programme implementation
and re-design where necessary.
Table A6.1: Evaluation Approach Per Member State

MS /Region Total Process Impact Both Other
AT 10 4 3 3
BE 18 9 5 3 0
BG 5 0 1 1 3
CY 2 0 0 0 2
CZ 35 5 3 0 27
DE 46 16 4 22 4
DK 16 10 0 0 6
EE 9 4 0 5
ES 16 12 2 0 2
FI 16 3 1 9 3
FR 9 7 1 1
GR 3 2 0 1
HU 26 5 16 0 5
IE 9 0 5 1 3
IT 45 28 6 2 9
LT 17 5 3 0 9
LU
MT 5 0 0 1 4
NL 3 0 0 3
PL 180 139 41 0
PT 12 6 3 3
RO 10 7 3 0
SE 18 5 2 0 4
SI 4 3 0 0 1
SK 7 5 0 0 2
UK 35 12 4 12 7
EU-27 556 287 103 67 91
page 75
Note: 1. Table shows count of evaluation approaches. Some evaluations listed in the Inventory do not
have an evaluation approach recorded (for example, if the evaluation is still in planning stage)
whilst others take multiple approaches and therefore have more than one approach recorded.
2. This data was extracted from the Inventories on 19 July 2012. The Inventories are updated on
an ongoing basis and therefore the numbers in this table are different from those in Table 8 which
were compiled using the data provided in the 27 CRs (which were finalised earlier in the year).
page 76
Table A6.2 sets out the geography of the evaluations across the Member States.
Clearly this is driven in large measure by the architecture of the OPs.
The most typical territorial level is national multi-regional accounting for 391 of
the evaluations.
There are 272 evaluations which are specifically regional.
Table A6.2: Territorial Level of Evaluations

MS /Region Total National Regional Other Unclear
multi-
regional
AT 13 4 9
BE 18 4 14
BG 5 5
CY 3 3
CZ 35 33 2
DE 66 31 35
DK 20 9 8 3
EE 11 11
ES 38 12 26
FI 14 14
FR 11 9 2
GR 13 13
HU 38 37 1
IE 9 9
IT 45 4 34 7
LT 21 21
LU
LV 2 2
MT 5 5
NL 3 3
PL 180 53 127
PT 12 9 3
RO 10 10
SE 18 18
SI 11 11
SK 18 18
UK 56 43 12 1
EU 27 675 391 272 7 5
page 77
Table A6.3 looks at the potential uses of the evaluations.
413 have potential operational value.
206 have potential strategic relevance.
Table A6.3: Potential Use of Evaluations

MS /Region Total Strategic Operational Mixed Unclear
AT 13 4 6 3
BE 18 17 1
BG 5 2 1 2
CY 3 2 1
CZ 35 1 33 1
DE 66 12 35 8 11
DK 20 11 5 4
EE 11 10 1
ES 38 9 29 0
FI 14 6 8
FR 11 6 5
GR 13 8 4 1
HU 38 33 5
IE 9 8 1
IT 45 14 30 1
LT 21 7 14
LU
LV 2 2
MT 5 4 1
NL 3 3
PL 180 42 138
PT 12 3 9
RO 10 3 3 4
SE 18 18
SI 11 2 1 8
SK 18 1 10 1 6
UK 56 6 49 1
EU 27 675 206 413 18 38
page 78
8 Conclusions
8.1 Positive messages

1. The key positive message is that by December 2011 a significant ESF A2E effort
had been mounted across the 27 Member States with:
Around 23 billion committed in total public sector costs (ESF + Member States).
Over 12.5 million final recipients engaged in ESF A2E key areas of
intervention, although there may be some multiple counting here.
Over 2.4 million employment results for final recipients.
2. There is a lot of evidence across the country reports that, in relation to A2E, there
have been important changes in an effort to respond to the economic crisis. This
has included such things as making much more support available to workless people
on the one hand but also trying to leverage additional employment through, for
example, incentives to employers to recruit the unemployed, as well as assistance for
the unemployed people moving into employment. This is further evidence to support
some of the key findings in the report on Evaluation of the Reaction of the ESF to the
Economic and Financial Crisis. It is important not to underestimate the challenge to
Managing Authorities in trying to implement OPs designed for the very different labour
market circumstances which prevailed pre-credit crunch.
3. There is a widespread appreciation that, particularly for clients with multiple
disadvantages (or multiple and complex needs), ESF A2E interventions need to reflect
the following:
Recipients may need interventions over a relatively long time period before
sustainable employment results can be achieved.
More disadvantaged recipients need access to a wider range of services
which go beyond narrowly defined A2E interventions. This includes services
such as addictions counselling, treatment of mental health issues, etc.
An essential design feature for A2E interventions, again for the more
disadvantaged, is a much more customised or individualised approach to the
package of interventions available for individuals.
There is an emerging appreciation of the importance of having a key
worker/personal adviser/work coach/etc who works closely with the individual
over the course of their journey towards and into employment.
4. There is a growing understanding of the importance of interventions which support
individuals once they have entered employment, particularly important in the
current economic crisis where labour market fragility is the norm. Having invested
significant ESF resources in getting individuals into employment it makes sense to
invest a bit more to maximise the long term value of that investment. However this
approach is not evident in sufficient Member States on the evidence of the country
reports.
8.2 Challenges for ESF A2E

5. As it is noted above, a very substantial number of individuals have been assisted by
ESF measures on A2E and have secured employment results. This is very important
given the nature and extent of the economic and labour market crisis confronting
Europe. In some instances however, experts have questioned the added value of
A2E ESF, particularly in terms of co-financing Member State mainstream welfare to
page 79
work programmes where many of the recipients are relatively short-term unemployed.
Notwithstanding the recession it is still the case that a significant portion of the newly
unemployed tends to find work without the need of intensive programmes of support.
This raises issues about the most appropriate allocation of ESF resources between
those further from and closer to the labour market.
6. There is not much evidence coming through the country reports of a serious
appreciation at governmental level within Member States of the need to see better
service integration to support A2E across public employment services, health
services, social services, etc. but this is essential to ensure that when interventions
and services reach individuals and families who are particularly disadvantaged there is
a joined up approach.
7. The treatment of CAV in many of the evaluations underpinning the country reports
appears to be superficial. When the senior officials responsible for ESF are at the point
of beginning to bargain for the resources required for the programme beyond 2013,
someone in the Commission will be asking about the CAV arising from the 76 billion
committed! Is it too late for the Commission to do something about this in terms of
guidance to Member States? It is in the interest of Member States to be able to secure
a continuing significant flow of ESF and so they should be prepared to commit some
evaluation resource to a serious estimate of CAV provided the EC gives them a clear
definition and some working examples of what is meant by CAV. Somehow or other
more robust information on the CAV associated with ESF A2E must be collated.
8. More generally, on the evidence of the country reports, the evaluation effort around
ESF A2E is fragmented and generally lacking in robustness. The fact that Mid-
Term Evaluations are no longer required means they have generally not occurred and
so there is undue dependence on what is essentially monitoring information provided
by beneficiary organisations. By this stage of the 2007-2013 Programme the
Commission should be able to inspect much more comprehensive and robust
information on the results of ESF A2E activity across the 27 Member States. This
finding needs to be seriously considered in the design of the new ESF programmes
post-2013.
9. If good quality counter-factual evaluations can be designed, resourced and launched
this is clearly of great value in understanding the cost effectiveness and overarching
CAV of ESF A2E. However as a minimum there is a requirement for evaluations which:
Independently measure job entry and job sustainability results in a
consistent and meaningful manner, making use of national social security data
bases where possible to provide confirmation.
Benchmark job entry and job sustainability results against Member State
A2E programmes where these exist and/or statistical information on the number
of unemployed people leaving benefits to go into work based on data held by
public employment services and/or social security agencies.
These types of evaluation would help to answer the basic question is ESF A2E
getting significant number of people into jobs and helping them sustain these jobs?
They would provide a firmer base for the various types of counter-factual evaluations
essential for demonstrating the impact and CAV of ESF A2E interventions.
page 80
Appendix: Detailed Tables

Table A2: Summary of Financial and Physical Progress to 2010
MS Public Final Recipients Employment Results Job Entry Job Sustainability Jobs Created
Sector Cost
Achieved Target % Target Achieved Target % Target Achieved Target % Achieved Target % Achieved Target %
Committed
Target Target Target
(Meuro)
AT1 128.2 65,578 70,000 93.7 34.9% 47.5% 73.4 34.9% 47.5% 73.4
2
BE 578.9 497,542 26-81% 26-81%
BG3 196.5 15,365 5,509 5,509
4
CY 37.9 2,979 5.9% 16.5% 35 5.9% 16.5% 35
CZ5 143.2 268,776 642,500 41.8 26,867 35,000 76.8
6
DE 1,018.3 131,657 7,455 7,455 1,955
DK7 30.3 14,939 88.0 2,424 14 2,424

EE8 80.8 80,225 11,699 11,699
ES9 6,179.1 4,379,545 6,533,824 67.0 947,201 2,978,730 31.8 947,201 2,978,730 31.8
10
FI 230.9 39,120 36.7% 36.7% 11,087
FR11 1,119.3 2,305,860 754,708 748,334
GR12 676.3 373,525 N/A
HU13 93.5 108,000 138,000 78.3 12.9% 28.5% 45.1 12.9% 28.5% 45.1
14
IE 1,051.8 213,346 59,996 59,996
IT15 1,711.8 311,400 21,250 21,250
16
LT 152.6 146,000 183,200 79.7 58,400 73,200 79.8 58,400 73,200 79.8
LU17 3.4 2,882 1,506 191.4 104 104
LV18 234.0 255,996 231,073 110.8 25-82% 30-40% 83-205% 25-82% 30-40% 83-205% 23% 24% 96%
MT19 16.3 5,740 715 715

NL20 74.1 33,176 21,180 156.6 13,965 13,965 6,950
PL21 4,537.6 1,086,399 2,149,000 50.6 18-45% 30-58% 44-119 18-45% 30-58% 44-119
PT22 229.4 64,854 124,088 52.3 13,367 11,600 115.2 13,367 11,600 115.2
RO23 13.7 38,288 235,000 16.3 N/A
page 81
MS Public Final Recipients Employment Results Job Entry Job Sustainability Jobs Created
Sector Cost
Achieved Target % Target Achieved Target % Target Achieved Target % Achieved Target % Achieved Target %
Committed
Target Target Target
(Meuro)
SE24 227.9 40,175 75,000 53.6 N/A
25
SI 161.3 85,912 205,900 41.7 56,700 56,700
SK26 143.2 260,030 23,319
27
UK 4,163.0 1,754,055 1,223,900 143.3 449,846 306,800 146.6 449,846 306,800 146.6
EU 23,233.3 12,581,364 2,403,339 2,403,339 6,950 63,228
Total
Note: Some employment results are reported in % terms (job entry rates) rather than absolute numbers in the evaluations that underpin the country reports.
General Note: yellow shading where no definition of employment results in country reports.
Country Notes:
1. Austria
Figures provided for PA2 (Combating unemployment). A2E activities are also undertaken under PA3b (Integration of people furthest from labour market). Data
is not available for PA3b but PA3 cost is 62.1 million, with 3,440 final recipients in 2010.
Employment results - Labour market status 3 months after end of measure.
2. Belgium
Figures estimated by Country Expert using range of assumptions as data only available at intervention level.
Cost Figures are Total Public Cost unclear whether Committed or some other measure.
Employment results - Labour market status 6 months after leaving (except BE-MS programme which is 3 months).
3. Bulgaria
Cost - Allocated spend (taken from OP HRD Operational Plan) for PA1 and PA2. Figure in brackets is Total Public Cost for PA1 (as reported in AIR 2010).
Final participants - Data for PA1 only. Data taken from AIR 2010.
Employment results Data for PA1 only. Measure is no. of people included in employment after successfully passing vocational or key competencies. Data
taken from AIR 2010.
4. Cyprus
Cost - Total Allocation of Funds (EU Contribution). Data taken from AIR 2010. Alternative figures provided in CR include Total Budget for OP EHCSC (149.7)
and Total Public Cost PA2 (49.9).
Final recipients Participants. Data taken from AIR 2010.
Employment results Job placement rate (people served by PES network). Data taken from AIR 2010.
5. Czech Republic
All figures taken from AIR 2010.
Cost Total Public Cost.
page 82
Final recipients Measure Number of persons who get any kind of support during the project realisation. Sum of figures for PA2, PA3 and OP Prague
Adaptability PA2.
Employment results Measure Number of jobs calculated as full time jobs created within the project (OP HRE PA2) and Number of jobs calculated as full
time jobs created for the persons vulnerable at the labour market within the project realization (OP HRE PA3).
6. Germany
Cost Calculated by TERU using following figures reported in CR. Table 3 - 2,686,837,172 allocated to A2E based on Priority Codes and Table 5 - 37.9%
cumulative commitments reported as a share of original allocation by Priority Codes presented within the OP planning documents.
Final recipient figures are based on 9 evaluations (out of 25) and employment results are based on 4 evaluations. As such these figures will be a significant
underestimate. Evaluation periods (e.g. to end June 2010, end September 2010, etc.) vary. No definition for employment results. Some % results are also
provided in main body of report.
7. Denmark
Data from AIR 2010.
Cost - Figures in country report given in Danish Krona. Converted using exchange rate (taken from FT.com) on 08/06/12 1DKK = 0.13450.
Employment results - Persons that have become employed, increased employability or increased competence levels.
8. Estonia
Cost Total Public Cost. Data from AMR 2011. Total Public Cost for 2007-13 157.2.
Final recipients Calculated by TERU using data from Table 3 Target groups and their participation in different measures of priority axis 3 Good quality and
long working life Finishing including cancelling for 2009 and 2010. Data also available for nos. starting 90,013. Data from AMR 2009 and 2010.
Employment results - Found employment and started work after participating in measures. Figures related to Measures 1.3.1 and 1.3.3 only (no data available
for 1.3.2, 1.3.4; and 1.3.5. Data from AMR 2011.
9. Spain
Figures drawn from AIR.
Cost Total Committed to 2010.
Employment results No definitions provided so uncertain what type of employment results.
10. Finland
Cost Data provided to Country Expert by Ministry of Employment and the Economy.
Final recipients - Participants in Priority II (Promoting access to employment and sustainable inclusion). Data from Strategic Evaluation (December 2011).
Employment results - Employed in open markets (26.9%) and employed with measures (9.8%) 6 months after end of measure. Figures for 3 months also
available. Relates to period 01/10/2009 to 30/09/2010. Data from MEE.
Job created Priority (Developing work organisations, labour force and entrepreneurship). Data from Strategic Evaluation (December 2011).
11. France
Cost ESF funding committed by 30/12/2010 for Mainland OP (data from AER 2011) and Total certified ESF support by December 2010 for OP La Reunion
(data from AIR 2010). Commitments for OP Priority Axis Increase participation to the labour market and decrease structural unemployment (considered by
Country Expert good proxy for A2E) are 2,516.9 (data from AIR 2010).
Final recipients Figures for OP Mainland France (AIR 2010) and OP La Reunion (Mid-term Evaluation covers period to end 2009).
Employment results OP Mainland France final recipients accessing employment (AIR 2010) plus OP La Reunion (no definition given) (AIR 2010 La
Reunion).
12. Greece
page 83
Commitments to OP HRD (data from OPs, AIRs and own work and analysis). No data available on commitments to OP Education and Lifelong Learning and
OP Administrative Reform. For information, Allocations/Payments for each OP are: OP HRD 1,039.4m/372.4m; OP E&LL 1,034.3m/184.2m; OP
Administrative Reform 8.5m/2.1m.
Final recipients - For OP HRD and OP E&LL; 2007-10. Data from AIRs, own data and analysis.
Employment results No data provided in CR.
13. Hungary
Cost Spend for OP SROP Priority 1 (A2E) to end 2010. Data from AIR 2010. Note: Data on Commitments not available.
Final recipients Involved recipients for OP SROP Priority 1 to end 2010. Data from AIR 2010.
Employment results - Employment 180 days post completion of assistance. Source of data unclear from text.
14. Ireland
Cost OP HCI total cost to end 2010.
Final recipients - Data from Mid-term evaluation HCI OP. Data also available from Review of Labour Market Programmes and Evaluation of LMAF but potential
overlap so excluded.
Employment results Into employment, education or training. Data from Mid-term evaluation HCI OP. Data also available from Review of Labour Market
Programmes and Evaluation of LMAF but potential overlap so excluded.
15. Italy
Cost - Includes all Regional and 2 (out of 3) National OPs. Data from AIR 2010.
Final recipients Includes Regional OPs only. Data from AIR 2010.
Employment results - Calculated by summing employment results in the evaluation reports provided in Annex of CR. Due to data gaps, this figure is likely to be
a significant underestimate. Definition of employment unclear.
16. Lithuania
All data taken from SFMIS.
All data to end 2010.
Cost Public Sector Cost Committed.
Employment results No definition provided.
17. Luxembourg
All figures given to 2010.
No source for data given but thought to be AIR 2010.
Employment Results No definition provided.
18. Latvia
Source: http://www.esfondi.lv
Cost Total Public Cost at 31.03.12.
Final recipients To 31.12.11. Includes Measures 1.3.1.1.3, 1.3.1.1.5; 1.3.1.2; 1.3.1.4, 1.3.1.5, 1.4.1.1.1 and 1.4.1.1.2.
Employment results To 31.12.11. Includes Measures 1.3.1.1.3, 1.3.1.2, 1.4.1.1.1 and 1.4.1.1.2. See below for definitions.
Job sustainability To 31.12.11. Includes Measures 1.3.1.1.3 The share of recipients employed within 6 months after the training, 1.4.1.1.1 Share of people
supported by complex integration measures economically active after 6 months and 1.4.1.1.2 Share of people supported with measures for specific target
groups who were economically active after 6 months.
Jobs created To 31.12.11. Includes Measure 1.3.1.2 Share of recipients who have started business within 6 months after the training for self -employed and
small businesses.
page 84
19. Malta
Data from AIR 2010. Data to end 2010.
Cost Total Public Cost Committed.
Employment results - No. of final recipients achieving an employment result or further study 6 months after receiving assistance.
20. Netherlands
Data from various evaluations.
Figures for 2007-09 - Action A (2007, 2008, 2008-2), Action B (2007, 2008, 2009) and Action C (2007, 2008).
Employment results No definition given.
Jobs sustained Sustained employment at 6 months (Actions A and C).
Different figures given for Action C in text compared to Annex 1 - Final recipients 26,871/6,430; Employment results 9,050/11,680.
21. Poland
Data from AIR 2010. Progress to 2010.
Cost Public Cost Committed.
Employment results No definition provided.
Targets (final recipients and employment results) to end of programming period.
22. Portugal
Data from AIR 2010. Figures for 2007-2010.
Employment results No definition provided. Figures for POPH only (Rumos and Pro-emprego not available).
23. Romania
Data from AIR 2010.
Final recipients - CR expresses concerns about accuracy of participant numbers for PA6 so this may be an overestimate.
Target data calculated by TERU based on figures in CR.
Employment results No data provided in CR.
24. Sweden
Data from AIR 2010.
Cost Total Public Cost (including public co-financing).
Final recipients Unclear from text but targets thought to be to 2013.
Employment results No data provided.
25. Slovenia
Source of data unclear although most likely AIR 2010.
Cost Total Public Cost on 31.12.2011 (ESF and SLO contribution).
Final recipients Unclear from text but targets thought to be to 2013.
Employment results Nos. remained/got employment after their contractual arrangements concluded. Includes some in employment when commenced
participation.
26. Slovak Republic
Data from AIR 2010.
Cost Total Public Cost (ESF).
page 85
Employment results - Jobs created by beneficiary organisations.
27. UK
Data from AIR 2010 (except HIPP AIR 2009).
Cost Public Sector Cost Committed.
Employment results Entering employment. Note Table 1 in UK CR presents data for England based on indicator "in employment 6 months following
completion" and "entering employment" for Scotland, Wales and NI. For consistency, the English job entry figures (taken from AIR 2010) have been used in this
report in place of the employment at 6 months.
page 86
Table A3: Types of intervention

MS Assess- Careers Personal Deving Vocation- Work ILMs/ Employ- Support Support Childcare Post job Support Training Inst. Others
ment of and devment key al skills exp./ social ment for self- to find and entry for up- for dev.
A2E labour and employ- devment intern- firms subsidies employ- work, job family support skilling existing (PES,
needs market confidenc ability ships ment/ search/ support (prog- employee etc.)
info., e building skills for start-up interview ression) s
advice work- skills
and place
guidance
AT 1
BE 2a 2b
BG 3
CY 4
5a
CZ 5b
6
DE
7
DK 8
9
EE
ES 10
FI 11
FR 12a 12b
13a 13b
GR
HU
IE 14
15a
IT 15b
LT 16a 16b
LU17a 17b
LV 18a 18b
page 87
MS Assess- Careers Personal Deving Vocation- Work ILMs/ Employ- Support Support Childcare Post job Support Training Inst. Others
ment of and devment key al skills exp./ social ment for self- to find and entry for up- for dev.
A2E labour and employ- devment intern- firms subsidies employ- work, job family support skilling existing (PES,
needs market confidenc ability ships ment/ search/ support (prog- employee etc.)
info., e building skills for start-up interview ression) s
advice work- skills
and place
guidance
MT 19
NL
PL 20
PT 21
RO22 22b
a
SE23a 23b
SI 24
25
SK
UK
EU 8 10 6 17 23 15 6 11 14 7 17 3 3 9 9 13
27
Notes: EU-12 countries shaded in blue.
Country Notes:
1. Austria Apprenticeships.
2a. Belgium Actions to improve access to work for women.
2b. Belgium Validation of skills and competencies.
3. Bulgaria Provision of social assistant/home helper.
4. Cyprus Support for women with caring responsibilities.
5a. Czech Republic Interventions being used unclear from country report.
5b. Czech Republic Priority Axis 3 included measures to improve access to employment of women access to childcare and care of dependent persons.
page 88
6. Germany Programmes to reconcile family life and work (e.g. enhancing child care, local networks for families).
7. Denmark Interventions being used unclear from country report.
8. Denmark Education and training.
9. Estonia Training of disabled childminders.
10. Spain Country report provides limited details on interventions. Others include ALMP and research scholarships for graduates. In addition, personal itineraries
mentioned in section on Effectiveness of Specific Interventions.
11. Finland Apprenticeships.
12a. France Reconciling work-life balance, women in employment.
12b. France Apprenticeships; Reinforced accompaniment of jobseekers; Workers mobility and outplacement; Sectoral economic restructuring; Innovative and
preventative HR initiatives; Local Employment and Inclusion Plans.
13a. Greece Including social entrepreneurship, cooperatives and social firms.
13b. Greece Measures to improve access to employment and increase sustainable participation and progress of women in employment to reduce gender based
segregation in the labour market and to reconcile work and private life, such as facilitating access to childcare and care for dependent persons.
14. Ireland Further education; second chance education; higher education.
15a. Italy - Measures to improve access to employment and increase sustainable participation of women in employment to reduce gender segregation in the labour
market and to reconcile work and private life.
15b. Italy Financial support for training courses; Research grants; Learning pathways within compulsory education; Training vouchers; Services vouchers; Abroad
mobility; Higher education; Permanent education; Services provided by PES; Job vouchers.
16a. Lithuania Measures for better adjustment for professional and family obligations; Instructing and consulting of employers with intention of provide working
conditions that would be more favourable to family life.
16b. Lithuania Maintenance of workplaces; Social rehabilitation services.
17a. Luxembourg No section in country report on interventions. Data drawn from text on target groups which indicated what type of interventions had been most
effective. Very little information provided.
17b. Luxembourg Objective to increase womens employment rate.
18a. Latvia Help getting childcare place.
18b. Latvia Support for taking qualification tests; Improving competitiveness; Special support for people with disabilities; Grants for travel and equipment; Services of
ergotherapist/signing translator (disabled young people with hearing or vision difficulties); Allowance for assistance services for 10 hours a week (disabled young people
with physical disabilities); Negotiate with employer (people with disabilities); Set up workplace to suit special requirements (people with disabilities).
19. Malta Targeted actions supporting and lifecycle approach to work; Training and support aimed at women; Facilitating families benefiting from childcare.
page 89
20. Poland Psychological and advisory support; Support for voluntary work; Support for initiatives to increase geographic mobility; Seasonal works.
21. Portugal Job clubs.
22a. Romania No information on interventions provided in country report. Lists priority intervention codes Modernisation of labour market institutions; Active and
preventative measures to support employment; Improving equal access to employment; Modernisation of Public Employment Service; Promoting active employment
measures; Promoting social inclusion; Promoting equal opportunities on the labour market; Active employment measures for unemployed and inactive persons including
persons from the rural areas involved in subsistence agriculture, managers and employed persons; Promotion of equal opportunities on the labour market, addressing
mainly women as a disadvantaged group; Transition from school to active life; Design, introduction and implementation of reforms in education and training systems.
22b. Romania Promotion of equal opportunities on the labour market, addressing mainly women as a disadvantaged group.
23a. Sweden No information on interventions available.
23b. Sweden Initiatives contribute to EES by focusing on 4 aims including Contribute towards the development of women and men who are furthest from the labour
market so they can start work or come closer to the labour market, and broaden opportunities for women and men to work on the basis of their pre-conditions.
24. Slovenia Education/training of employed people; Preparation of vocational standards; Training of employees temporarily waiting for work; competence centres for
HRD; Company scholarship schemes; Co-financing of PhD studies; Subsidies for shorter working hours; Slovene qualification framework; Co-financing projects of social
partners for increasing working conditions; Employment of long-term unemployed people, older unemployed people, young unemployed people and hard-to-employ
people; Institutional training for national vocational framework; On-the-job training (including of candidates for graduation); Development of aids for lifelong career
orientation.
25. Slovak Republic No information on interventions available.
26. 8 specifically note childcare as support; 8 implied.
page 90
Table A4: Target Groups

NEET and
complex
needs
AT 1
BE
BG 3 2
4 5
CY
CZ 6 7
DE
8 9
DK
10 12a 11
EE 12b
13 14
ES
FI15
FR
GR
HU 17 18 19
IE 20
21 23a 23b 22
IT
24 25
LT 26
LU 27
LV 29 28a 28b
30
MT 31
32
NL 33 34
page 91
NEET and
complex
needs
PL 35 36
37
PT
38 39 39
RO 40
SE 41 42 43
44a 44b
SI
SK45
UK 46
EU 27 18 14 10 17 19 4 14 5 5 11 3 12 15 13
Notes: EU-12 countries shaded in blue.
Country Notes:
1. Austria Low qualified.
2. Bulgaria Including school and university students.
3. Bulgaria Employees.
4. Cyprus Vulnerable groups.
5. Cyprus Over 30s.
6. Czech Republic Disadvantaged people.
7. Czech Republic - War veterans; Persons taking care of dependent person; Persons before/after release from prison; Young people in care; Parents on or returning
from maternity/ paternity leave; Job applicants; Employers; Institutions, social partners and cooperating organisations.
8. Denmark Including inactive.
9. Denmark Employed persons.
10. Estonia Including inactive.
11. Estonia Including persons who do not know the Estonian language (i.e. ethnic Russian).
12a. Estonia Employees.
12b. Estonia Families of disabled persons; persons released from prison; Other vulnerable persons; Self-employed.
13. Spain Country report notes that information on target groups in evaluations is scarce.
14. Spain Vulnerable groups.
page 92
15. Finland No details of target groups provided.

16. France Target groups with special inclusion needs; Deprived urban areas.
17. Hungary Including mental illness.
18. Hungary Groups facing special disadvantages such as the homeless, those living in and leaving child protection institutions, convicts serving imprisonment or
leaving it.
19. Hungary Low educational attainment.
20. Ireland Low qualifications.
21. Italy Including inactive.
22. Italy Disadvantaged groups.
23a. Italy People in employment (esp. in response to crisis).
23b. Italy Graduates.
24. Lithuania Those under threat of redundancy.
25. Lithuania Vulnerable females.
26. Lithuania Convicts and ex-convicts; Addicted persons; People from rural areas retreating from agricultural activities.
27. Luxembourg Persons with a large distance to the labour market.
28a. Latvia Young people aged 18-24 with basic education but without work experience; young people with disabilities.
28b. Latvia Young unemployed university graduates (up to age 25).
29. Latvia People of high risk of losing their job because they lack specific skills (in wholesale and retail trade, care and motorbike service and processing industry).
30. Malta Country report presents % of final recipients in each of these groups but no evidence they were stated priorities for OP.
31. Malta Disabled, illiterate persons, persons with mental health difficulties, ex-convicts, youths in institutional care, ex-substance users and women in difficulty.
32. Netherlands Including students enrolled in professional education and special secondary education.
33. Netherlands People with large distance to the labour market.
34. Netherlands Prisoners, people on a hospital order, juvenile delinquents.
35. Poland In particular women returning to work after maternity leave.
36. Poland Persons taking care of child younger than 18; Unemployed residing in rural/urban municipalities intentioning to take up employment within areas not related
to plant/animal production; unemployed former prisoners.
37. Portugal Vulnerable population.
38. Romania Includes inactive.
39. Romania Students, undergraduates, graduates, apprentices and tutors.
40. Romania Unemployed/inactive persons from rural areas involved in subsistence agriculture, managers and employed persons.
41. Sweden Long term sick.
42. Sweden Furthest from the labour market.
43. Sweden Men.
44a. Slovenia Employed people (to retain employment).
44b. Slovenia Students (pre-graduation).
45. Slovak Republic No information on target groups available.
46. UK People subject to multiple deprivations.
page 93
Table A6: Effectiveness of ESF A2E Interventions
% Recipients Cost Per % of Cost Per
Achieving Employment Recipients Other
Employment Result Achieving Progression
Result Other
Progression
AT (Austria)
A2E Aggregate 34.9
Apprenticeships (PA2) 74
SOB/GBPs1 (PA2) Generally <20
Production schools 302
(PA2)
PA3b3 9 ( first labour 7 (other
market) vocational
15 (second training)
labour market
transitional
employment)
Young people (PA2) 67
Males 594
Females 714
Production schools
(PA2)
With school 35
leaving certificate
Without school 23
leaving certificate
Belgium (BE)5
A2E Aggregate Range 26-81
Vocational training (BE- 32
VL)
Job training and 39.8
coaching (BE-VL)
Females (BE-B) 386
Under 25s (BE-B) 37.66
Bulgaria (BG)
A2E Aggregate 35.9% 35,669
Cyprus (CY)
A2E Aggregate 16.5
PES 5.97
PES young people 8.57
Czech Republic (CZ)
A2E Aggregate
page 94

Result Other
Progression
National individual 7,7868
projects
Grant projects 16,2768
Regional individual 50,5138
projects
Germany (DE
A2E Aggregate 5.7 136,593
Support for start-up c.90% survival
at 3 years
Denmark (DK)9,10
Developing human 12,987
resources
Establishing and 11,042
developing new firms
Estonia (EE)
Measure 1.3.1 24
Increasing availability
of qualified labour force
Measure 1.3.3 Active 40
labour market
measures supporting
welfare measures
Spain (ES)
Finland (FI)
A2E Aggregate 36.711 7.8 12
Employment subsidy to 51.011 1.012
state, employment to
ELY (Regional MEE
office)
Wage subsidy total 37.011 3.012
11
Apprenticeship total 58.0 2.012
Start-up money for 69.011 0.412
companies
Employment benefit in 35.911 6.312
labour market coaching
Subsidised 45.811 2.312
employment total
page 95
Result Other
Progression
Vocational labour 36.911 6.412
market training
Total subsidised 41.111 4.512
employment +
vocational LM training
Coaching labour 30.411 2.612
market training
France (FR)
A2E Aggregate13 32.7 1,483 68.314
15
Local employment and 34
inclusion plans
16
Job coaching in Seine 37
Maritime
Greece (GR) No data
available
A2E Aggregate
Hungary (HU)
A2E Aggregate 12.917 -
Ireland (IE)
A2E Aggregate 18 28.1 13,649
Labour market 49 48,948 16 153,202
programmes 19
Labour market 29 into 4321
activation fund 20 employment
13 into self-
employment
Specific skills training22 58 13,64023
Traineeship22 72 20,64023
Community Training 41 53,27123
Centres22
Italy (IT)
Lithuania (LT)
ALMP Assistance to 36
acquire professional
skills24
ALMP Employment 30
subsidies24
ALMP Public works24 14
page 96

Result Other
Progression
ALMP Vocational 18
training24
Rural labour force 625
change of focus from
agriculture to other
activities
Luxembourg (LU)
1.1 Increase 86 16,315
employment rate of
older workers
1.2 Increase womens 49 27,096
employment rate
1.3 Facilitate insertion 5 54,523
of young people in the
labour market
1.4 Activate persons 45 29,425
with a large distance to
the labour market
Latvia (LV)
A2E Aggregate Range 25-82 -
Training of unemployed 21.8
and job seekers
Phases 1 and 2 26
Training of unemployed 95
and job seekers
Phase 3 (training and
internships for
unemployed graduates
25 and under)
Malta (MT)
27
PA3 (Promoting an 12.5 5.428
equal and inclusive
labour markets)
interventions and
projects
Training and support 029 52.630
for women
Support for vulnerable 1127 3031
groups
page 97
Result Other
Progression
Employment aid 9432 14,395
scheme (wage subsidy
to employers to
incentivise recruitment
from
disadvantaged/disabled
groups)
Netherlands (NL)
33,34
Action A (projects that 5
equip people with large
distance to labour
market with skills and
increase prospects for
sustainable integration)
Action B (projects that 1633 6235
improve the position of
prisoners, people on a
hospital order and
juvenile delinquents on
the labour market)
Action C (programmes 71.433
that promote the
inclusion into the labour
market of students
enrolled in professional
education and special
secondary education)
Poland (PL)
A2E Aggregate Range 18-45
Priority I, SO 2: To 20.8
broaden the impact of
ALMP
Priority VI, SO1: 45.3
Increasing coverage of (employed)
Labour Market Active 19.8 (self-
Policy employed)
Priority VI, SO2: 39.7
Increasing employment (employed)
among young people 8.36 (self-
employed)
page 98

Result Other
Progression
Priority VI, SO3:
Reduction of
unemployment
amongst people
disadvantaged in
labour market
Women N/A
LTU 31.4
Persons with 33.1
disabilities
Unemployed 41.8
living in rural
areas
Priority VI, SO4: Higher 35.6
employment among (employed)
older people 18.8 (self-
employed)
Priority VII, SO1: Better 17.6
access to labour
market for people
threatened by social
exclusion
Portugal (PT)
POPH (Norte, Centro, 12,90237
Alentejo, Lisboa and
Algarve)
Rumos (Madiera) 7,03137
Romania (RO)
A2E Aggregate
KAI 5.2 Active 1.89
employment measures
for persons from rural
areas involved in
subsistence agriculture,
managers and
employed persons
Sweden (SE) No data
available
A2E Aggregate
Slovenia (SI)
Job subsidies 60.0 (estimate)
page 99
Result Other
Progression
Persons already 100
employed
Developing new 63 (including
employment employed and
opportunities self-employed)
Unemployed persons 55
Slovak Republic (SK)
A2E Aggregate38
United Kingdom (UK)
A2E Aggregate39 25.6 9,254
Note: Cells shaded in green have been calculated by authors using data from Table 2 (i.e. % Recipients
Achieving Employment Result = Employment Results Achieved / Final Recipients Received; Cost per
Employment Result = Public Sector Cost Committed / Employment Results Achieved). Given gaps in data,
care must be taken in interpreting these results.
Country Notes:
1. Austria SOBs/GBPs involve independent job search, punctuality, self-confidence, psychological

stability, realistic self-assessment, coping with an eight hour day, more motivation in looking for work,
higher social integration.
2. Austria Directly after the measure.
3. Austria PA3b involves combination of innovative counselling methods coming from social work field
combined with employment measures.
4. Austria After 36 months. Difference between genders can partly be explained by military service for
males.
5. Belgium Figures also available for individual OPs.
6. Belgium Compared to overall figure of 40.7%.
7. Cyprus % of people served achieving job placement.
8. Czech Republic Cost of new created jobs (not employment).
9. Denmark Figures in country report given in Danish Krona. Converted using exchange rate (taken
from FT.com) on 08/06/12 1DKK = 0.13450.
10. Denmark Persons achieving employment, increasing employability or increasing competence levels.
11. Finland Employed in open markets or with measures: 6 months after end of measure. Figures for 3
months also available.
12. Finland In labour market training 6 months after end of measure. Figures for 3 months also available.
13. France To end 2010. Final recipients accessing employment for Mainland France; no definition for
La Reunion.
14. France To end 2010. Positive exits including accessing employment.
15. France Figures also available for women and 50+ but unclear if % of women/50+ accessing
employment or % of accessing employment which are women/50+.
16. France Accessing permanent job.
17. Hungary Unclear if data for Priority 1 or SROP as whole; Employment 180 days post completion of
assistance.
18. Ireland Into employment, education or training, 2007-10 (Data from Mid-term Evaluation of HCI OP).
19. Ireland 2007-08.
20. Ireland 2010 June 2011.
21. Ireland Progression into further education and training.
22. Ireland Figures for 2007.
23. Ireland Cost per person progressing into further training, education and/or employment.
24. Lithuania Taken from evaluation of the measure Integration of Job-seekers into the Labour Market.
25. Lithuania - % of participants who switched to non-agricultural activities after training.
26. Latvia Figures for 1.3.1.1.3 Training of unemployed and job seeker Phase 1 (01.09.09 31.12.10)
which provided modular training, professional training, re-qualification, improved qualifications, support
page 100
for taking qualification tests, training at a new workplace, improving competitiveness and special
support for people with disabilities. % of recipients employed within 6 months after the training.
27. Malta - % of final recipients achieving employment result or engaged in further education (by end
2010).
28. Malta - % of final recipients achieving a soft result such as increased personal capacity or other
progress towards employment not covered by hard results (by end 2010).
29. Malta Too early to measure employment at 6 months at end 2010.
30. Malta - % of final recipients gaining soft result, a qualification or certificate of attendance.
31. Malta - % of final recipients gaining a qualification or certification in capacity building.
32. Malta - % retained in employment after end of placement.
33. Netherlands Achieved employment results.
34. Netherlands Country expert argues employment outcomes low as most projects had not completed
at time of analysis (2007, 2008 and 2008-2).
35. Netherlands - % obtaining qualification or partial qualification (e.g. diploma).
36. Portugal Figures for POPH only (Rumos and Pro-emprego not available), 2007-2010.
37. Portugal Figures for 2010.
38. Slovak Republic Aggregate data only available for jobs created by beneficiary organisations.
39. UK Some data for nations/regions (e.g. West Wales) also available.
page 101
Table A7. Evaluation data sources
AT BE BG1 CY2 CZ1 DE DK EE ES FI1 FR GR1 HU1 IE1 IT1 LT1 LU LV2 MT3 NL PL PT RO1 SE SI SK UK Total
Final 7 3 2 - 3 13 5 2 5 1 4 4 6 2 14 3 0 - 0 0 11 0 2 0 0 0 8 95
recipient
contact
data
Financial 3 4 2 - 2 21 7 1 9 3 3 4 2 3 14 3 1 - 4 1 5 6 3 1 1 1 8 112
data
Output 5 4 3 - 2 25 3 3 9 1 3 4 7 4 16 3 0 - 4 1 13 5 3 1 1 1 10 131
data
Results 5 7 2 - 3 15 3 3 9 2 3 4 6 4 7 3 1 - 4 1 14 3 3 1 0 1 9 113
data
based on
monitoring
Results 6 5 2 - 1 18 0 2 0 1 1 0 5 1 8 1 0 - 0 0 15 0 2 0 0 0 14 82
data
based on
research
Notes:
1. Calculated by TERU for Bulgaria, Czech Republic, Finland, Greece, Hungary, Ireland, Italy, Lithuania and Romania based on individual evaluation information
provided in Annex.
2. Latvia and Cyprus information provided in Annex confusing so not included here.
3. Malta Table provided is for All AIRs reporting data. Country experts accessed AIRs for 2007-2011, so attached figure of 4 to each
page 102
Table A8. Evaluation approaches and methodologies

Management Results Results Results Impact Impact Evaluation
and Using based on final based upon Evaluation Evaluation
Implementation Monitoring recipient Case sample final
Processes Data14 Study / Focus recipient
Group15 research16
Evaluation (AT 0) (BE 4) (AT 0) (BE 8) (AT 0) (BE 1) (AT 0) (BE 1) (AT 0) (BE 0) (AT 0) (BE 2) (AT 0) (BE 0)
specifically (DE13 ) (DK 0) (DE 13) (DK 0) (DE 3) (DK 0) (DE 7) (DK 0) (DE 3) (DK 0) (DE 0) (DK 0) (DE 2) (DK 0)
focusing on (ES 10) (FR 3) (ES 12) (FR 3) (ES 7) (FR 2) (ES 1) (FR 0) (ES 2) (FR 0) (ES 0) (FR 0) (ES N/A3) (FR
A2E (GR 5) (HU 0) (GR 6) (HU 3) (GR 0) (HU 0) (GR 0) (HU 1) (GR 5) (HU 0) (GR 0) (HU 2) 0) (GR 0) (HU
interventions (IE 1) (IT 0) (LT (IE 2) (IT 2) (LT (IE 1) (IT 0) (LT (IE 2) (IT 1) (LT (IE 0) (IT 0) (LT (IE 2) (IT 0) (LT 0) (IE 1) (IT 0)
and target 1) (LV 1) (MT 1) (LV 0) (MT 4) 1) (LV 0) (MT 0) 1) (LV 1) (MT 0) 0) (LV 0) (MT 0) 0) (LV 1) (MT 0) (LT 1) (LV N/A3)
groups 44) (NL 0) (PL (NL 0) (PL 1) (NL 0) (PL 0) (NL 0) (PL 1) (NL 0) (PL 1) (NL 0) (PL 0) (MT 0) (NL 0)
0) (PT 05) (SE (PT 0) (SE 1) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PL 0) (PT 0)
N/A6) (SI 0) (SK (SI 0) (SK 1) (SI 0) (SK 0) (SI 0) (SK 0) (SI 0) (SK 0) (SI 0) (SK 0) (SE 0) (SI 0)
0) (UK 8) (Cell (UK 3) (Cell (UK 2) (Cell (UK 7) (Cell (UK 6) (Cell (UK 2) (Cell (SK 0) (UK 1)
total 50) total 60) total 17) total 23) total 17) total 9) (Cell total 5)
specifically (DE 6) (DK 0) (DE 6) (DK 0) (DE 6) (DK 2) (DE 8) (DK 2) (DE 3) (DK 0) (DE 0) (DK 0) (DE 6) (DK 0)
focusing on (ES 4) (FR 0) (ES 4) (FR 0) (ES 2) (FR 0) (ES 0) (FR 0) (ES 0) (FR 0) (ES 0) (FR 0) (ES N/A3) (FR
subset of A2E (GR 5) (HU 1) (GR 6) (HU 3) (GR 0) (HU 2) (GR 0) (HU 1) (GR 5) (HU 3) (GR 0) (HU 0) 0) (GR 0) (HU
interventions (IE 0) (IT 0) (LT (IE 1) (IT 1) (LT (IE 0) (IT 0) (LT (IE 0) (IT 4) (LT (IE 0) (IT 0) (LT (IE 0) (IT 1) (LT 1) (IE 0) (IT 0)
and target 1) (LV 0) (MT 4) 1) (LV 0) (MT 4) 0) (LV 0) (MT 0) 1) (LV 0) (MT 0) 1) (LV 0) (MT 0) 0) (LV 0) (MT 0) (LT 0) (LV N/A3)
groups (NL 0) (PL 2) (NL 1) (PL 9) (NL 1) (PL 5) (NL 0) (PL 8) (NL 0) (PL 7) (NL 0) (PL 0) (MT 0) (NL 0)
(PT 5) (SE (PT 5) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 1) (SE 0) (PL 1) (PT 0)
focused on (DE 0) (DK 0) (DE 0) (DK 7) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0)
14 i.e. evaluation that is based upon administrative resources rather than research.
15 i.e. evaluation based upon very small scale final recipient research and useful for investigating qualitative issues (may form part of a more broadly-based evaluation).
16 i.e. evaluation research through representative final recipient survey examining the gross results.
page 103
Management Results Results Results Impact Impact Evaluation
and Using based on final based upon Evaluation Evaluation
Implementation Monitoring recipient Case sample final
Processes Data14 Study / Focus recipient
Group15 research16
target group, a (ES 4) (FR 0) (ES 5) (FR 0) (ES 4) (FR 0) (ES 1) (FR 0) (ES 0) (FR 0) (ES 0) (FR 0) (ES N/A3) (FR
subset of which (GR 5) (HU 1) (GR 6) (HU 1) (GR 0) (HU 1) (GR 0) (HU 0) (GR 6) (HU 1) (GR 0) (HU 0) 0) (GR 5) (HU
includes A2E (IE 0) (IT 0) (LT (IE 0) (IT 2) (LT (IE 0) (IT 0) (LT (IE 0) (IT 2) (LT (IE 0) (IT 0) (LT (IE 0) (IT 0) (LT 0) (IE 0) (IT 0)
final recipients. 1) (LV 0) (MT 0) 1) (LV 0) (MT 0) 1) (LV 0) (MT 0) 1) (LV 1) (MT 0) 0) (LV 0) (MT 0) 1) (LV 0) (MT 0) (LT 0) (LV N/A3)
(NL 0) (PL 1) (NL 0) (PL 3) (NL 0) (PL 2) (NL 0) (PL 3) (NL 0) (PL 3) (NL 0) (PL 0) (MT 0) (NL 0)
(PT 05) (SE (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PL 0) (PT 0)
6
N/A ) (SI 0) (SK (SI 0) (SK 0) (SI 0) (SK 0) (SI 0) (SK 0) (SI 0) (SK 0) (SI 0) (SK 0) (SE 0) (SI 0)
which focuses (DE 0) (DK 7) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0) (DE 0) (DK 0)
on intervention (ES 2) (FR 0) (ES 3) (FR 1) (ES 1) (FR 1) (ES 0) (FR 0) (ES 2) (FR 0) (ES 0) (FR 0) (ES N/A3) (FR
type where (GR 5) (HU 0) (GR 6) (HU 0) (GR 0) (HU 0) (GR 0) (HU 0) (GR 5) (HU 0) (GR 0) (HU 0) 0) (GR 0) (HU
subset of the (IE 0) (IT 8) (LT (IE 0) (IT 5) (LT (IE 0) (IT 1) (LT (IE 0) (IT 3) (LT (IE 0) (IT 0) (LT (IE 0) (IT 0) (LT 0) (IE 0) (IT 0)
target group are 0) (LV 0) (MT 4) 0) (LV 0) (MT 4) 0) (LV 0) (MT 0) 0) (LV 0) (MT 0) 0) (LV 0) (MT 0) 0) (LV 0) (MT 0) (LT 0) (LV N/A3)
A2E final (NL 0) (PL 2) (NL 0) (PL 2) (NL 0) (PL 2) (NL 0) (PL 2) (NL 0) (PL 3) (NL 0) (PL 0) (MT 0) (NL 0)
recipients. (PT 05) (SE (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PT 0) (SE 0) (PL 0) (PT 0)
Total 148 165 57 70 63 14 18
Notes:
1. Summary table not provided for:

Bulgaria
Cyprus
Czech Republic
Finland
Luxembourg
page 104
Romania
2. Belgian summary table does not include row for Evaluation specifically focusing on subset of A2E interventions and target groups so figures of zero have been
allocated to each cell in this row for Belgium.
3. Spain/Latvia do not provide any information for Other impact evaluation column.
4. Malta Table provided is for All AIRs reporting data. Country experts accessed AIRs for 2007-2011, so attached figure of 4 to each category.
5. Portugal summary table only contains row Research specifically focusing on subset of A2E interventions and target groups so figures of zero have been allocated to
each cell in the other rows.
6. Sweden Data for Evaluation/analysis of management and implementation process unclear.
7. Estonia has only provided summary data for each column so not included in table. For reference, 1 evaluation of management and implementation processes; 3
evaluations of results using monitoring data; 1 evaluation of results based on final recipient case study/focus group; 2 evaluation of results based upon sample final
recipients; no theory-based, counter-factual or other impact evaluations.
page 105
Glossary
Access to Employment (A2E) is one of the key policy fields for the 2007-2013 ESF
Operational Programmes (OPs) and is described as: enhancing access to
employment and the sustainable inclusion in the labour market of job seekers and
inactive people, preventing unemployment, in particular long term and youth
unemployment, encouraging active ageing and longer working lives and increasing
participation in the labour market (Art. 3 of the ESF Regulation No 1081/2006).
Beneficiary: an operator, body or firm, whether public or private, responsible for
initiating or initiating and implementing operations. In the context of aid schemes under
Article 87 of the Treaty, beneficiaries are public or private firms carrying out an
individual project and receiving public aid (Art. 2 of the ESF Regulation No 1081/2006).
Community Added Value: The definition used is based on the methodological note
prepared by the Evaluation and Impact Assessment Unit in October 2008 which
identifies 4 key dimensions of Community Added Value (CAV). CAV measures the
difference made by ESF A2E funding and programmes compared to the efforts of
Member States alone. There are four key dimensions: volume effects, scope effects,
role effects and process effects.
Volume effects: ESF action adds to existing action, either by supporting
national action in general (mirroring) or specific areas of national policy
(boosting).
Scope effects: ESF action broadens existing action by supporting groups or
policy areas that would not otherwise receive support
Role effects: ESF action supports local/regional innovations that are taken up at
national level or national innovative actions that are then mainstreamed
Process effects: ESF action influences Member States administrations and
organisations involved in the programmes.
Final Recipient: 'Final recipients' refer to participants (i.e people) in supported
operations. These are to be clearly distinguished from beneficiaries. Beneficiary is
defined in Art. 2(4) of Council Reg. (EC) No 1083/2006 as: "an operator, body or firm,
whether public or private, responsible for initiating or initiating and implementing
operations. In the context of aid schemes under Article 87 of the Treaty, beneficiaries
are public or private firms carrying out an individual project and receiving public aid".
Beneficiary can e.g. be a NGO implementing an ESF-funded project providing services
for final recipients (participants).
Intermediate body: any public or private body or service which acts under the
responsibility of a managing or certifying authority, or which carries out duties on behalf
of such an authority vis--vis beneficiaries implementing operations (Art. 2 of the ESF
Regulation No 1081/2006).
Operation: a project or group of projects selected by the managing authority of the
operational programme concerned or under its responsibility according to criteria laid
down by the monitoring committee and implemented by one or more beneficiaries
allowing achievement of the goals of the priority axis to which it relates (Art. 2 of the
ESF Regulation No 1081/2006).
Operational Programme: document submitted by a Member State and adopted by the
Commission setting out a development strategy with a coherent set of priorities to be
carried out with the aid of a Fund, or, in the case of the Convergence objective, with
page 106
the aid of the Cohesion Fund and the ERDF (Art. 2 of the ESF Regulation No
1081/2006).
Priority Axis: one of the priorities of the strategy in an operational programme
comprising a group of operations which are related and have specific measurable
goals (Art. 2 of the ESF Regulation No 1081/2006).
Public expenditure: any public contribution to the financing of operations whose origin
is the budget of the State, of regional and local authorities, of the European
Communities related to the Structural Funds and the Cohesion Fund and any similar
expenditure. Any contribution to the financing of operations whose origin is the budget
of public law bodies or associations of one or more regional or local authorities or
public law bodies acting in accordance with Directive 2004/18/EC of the European
Parliament and of the Council of 31 March 2004 on the coordination of procedures for
the award of public works contracts, public supply contracts and public service
contracts shall be regarded as similar expenditure (Art. 2 of the ESF Regulation No
1081/2006).
Sub-priority axis: Level of implementation below Priority Axis in the Operational
Programmes.
page 107

Final Synthesis Report On Access To Employment 12 Oct 2012

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Final Synthesis Report On Access To Employment 12 Oct 2012

Загружено:

Авторское право:

Доступные форматы

Final Synthesis Report on Access to

ESF Expert Evaluation Network

Final Synthesis Report on Access to Employment

ESF Expert Evaluation Network

Tables and Figures ....................................................................................................... 5

1 Executive Summary .............................................................................................. 6

2 Background and study objectives .................................................................... 12

3 Overall progress in delivery of ESF Access to Employment ......................... 23

4 Analysis of interventions and target groups ................................................... 29

5 Factors influencing and good practice in implementation ............................. 46

6 Community Added Value ................................................................................... 56

7 Evaluation data, approaches and methodologies ........................................... 62

Annex to Section 7: Analysis of evaluations based on Inventory ......................... 74

Appendix: Detailed Tables ......................................................................................... 81

Glossary ..................................................................................................................... 106

Tables and Figures

Table 1. Number and Volume of ESF OP by Member State ........................................ 23

Figure 1. % Change in Employment Levels (Aged 15-64), 2007 to 2011..................... 18

1.1 Background and study objectives

1.2 Overall progress in delivery of ESF Access to Employment

1.3 Analysis of interventions

1.4 Factors influencing and good practice in implementation

Good practice in implementation was highlighted across a wide range of country

1.5 Community Added Value

1.6 Evaluation data, approaches and methodologies

Challenges for ESF A2E

2.1 Background to Report

2.2 ESF and Enhancing Access to Employment (A2E)

The modernisation and strengthening of labour market institutions, in particular

2.3 Crisis in economy and labour market

Employment levels of people with disability improved up to 2000 but then

Reacting to the Recession

The Member States responded quickly to the economic crisis.

Variations in Labour Market Impacts of Recession

Figure 1. % Change in Employment Levels (Aged 15-64), 2007 to 2011

Source: Eurostat Labour Force Survey

Figure 2. % Change in Part-Time Employment (Aged 15-64), 2007 to 2011

Source: Eurostat Labour Force Survey

Unemployment Across EU Member States

Source: Eurostat Labour Force Survey

Overview of Labour Market

2.4 This Synthesis Report

Methodological Approach and Reporting Process

3 Overall progress in delivery of ESF Access to Employment

2007-2013 ESF Operational Programmes

Table 1. Number and Volume of ESF OP by Member State

Overview of Financial and Physical Progress in Relation to ESF A2E

Good Volumes Overall

Member Public Sector Final Recipients Employment Result

BE 578.9 497,542 26-81%

BG 196.5 15,365 5,509

CY 37.9 2,979 5.9% 16.5% 35

CZ 143.2 268,776 642,500 41.8

DE 1,018.3 131,657 7,455

DK 30.3 14,939 17,000 88.0 2,424 17,300 14

EE 80.8 80,225 11,699

ES 6,179.1 4,379,545 6,533,824 67.0 947,201 2,978,730 31.8

FI 230.9 39,120 36.7%

FR 1,119.3 2,305,860 754,708

GR 676.3 373,525 N/A

HU 93.5 108,000 138,000 78.3 12.9% 28.5% 45.1%

IE 1,051.8 213,346 59,996

IT 1,711.8 311,400 21,250