Вы находитесь на странице: 1из 35

M-Tool SET

CONTENTS

Section I
NA Context to Develop Monitoring Visits to LLP Partnerships 2
Section II
M-Tool Strategy for Monitoring Educational Partnerships 4
Section II. A
M-Tool Strategy - Methodology 4
Section II. B
M-Tool Strategy - Tools 5
1. NA Memo 5
2. Letter to the institution 6
3. Qualitative techniques for collecting data 7
4. Summary of the Interview Guide 10
5. General interview Guide 15
6. Guide for analysis of monitoring data 31
Example of the analysis grid for the interview 32
Example of the analysis grid for observation 32
7. NA Monitoring report 33
8. Follow-up report 35

1 / 35
M-Tool SET

Section I

NA CONTEXT TO DEVELOP MONITORING VISITS TO LLP PARTNERSHIPS

1. Follow the Guide for National Agencies (NA) established by the European Commission (EC)
Main topics:
1.1. European definition of concepts in the Lifelong Learning Programme (LLP) – LLP Glossary1.
1.2. On-going monitoring of projects - NA will monitor project implementation at any moment during the project lifetime (in situ
project visits).
1.3. Pre-defined criteria to select beneficiaries in a balanced spread: institution size, institution type, geographical criteria, grant
level.

Monitoring:
ƒ Monitoring is the regular observation and recording of activities taking place in a project or programme. It is a process of routinely
gathering information on all aspects of the project. To monitor is to check on how project activities are progressing. It is
observation; systematic and purposeful observation. Monitoring also involves giving feedback about the progress of the project to
the donors, implementers and beneficiaries of the project. Reporting enables the gathered information to be used in making
decisions for improving project performance. in “LLP 2007-2013 Guide for Applicants”
ƒ Follow-up of performance and realisations of a beneficiary and provision of feedback and guidance in view of improving the
supported activity and management of the grant as needed. in “Guide for National Agencies implementing the LLP”
Monitoring (at project level): The process involves continuous and systematic control of the project’s progress. The intention is manage
and if necessary correct any deviation from the operational objectives and thus improve the performance. Every project should be
monitored throughout its duration in order to ensure its success. Monitoring consists of supervision of activities, comparison with the
work plan and using the information obtained for the improvement of the project. During the monitoring process dissemination and
exploitation activities must be carefully checked, verified and, if necessary - reoriented and adapted. in “LLP 2007-2013 Guide for
Applicants”
Thematic Monitoring: Thematic Monitoring is a qualitative process put in place to increase the impact of the LLP programme. The
main elements are: 1) Clustering of projects into thematic groups to gain overview on specific contents and outcomes; 2) Facilitating
exchange of experience between project actors with a view to improving quality and impact at individual project level; 3)
Facilitating the networking of projects, practitioners and decision makers with a view to the future orientation of political priority and
strategy. in “LLP 2007-2013 Guide for Applicants”

2 / 35
M-Tool SET

2. Establish a national monitoring strategy according to the EC orientations:


2.1. Structured approach considering, at national level, the objectives and target groups of the LLP sectoral programme (national
criteria to select projects for monitoring).
2.2. NA Purpose:
• Collecting information on the quality in management of the LLP Sectorial Programme and the efficiency of the
implementation of the project in the institution / organization and in the educational community.
2.3. NA Aims:
• To collect information regarding the quality in management of the projects.
• To support the beneficiaries in project implementation.
• To lead the beneficiaries to possible solutions of the problems detected in the implementation.
• To be familiar with the real resources (human, material and institutional) applied to the projects.
• To assess the progress of the projects.
• To disseminate examples of good practices.
• To establish a close relation between the beneficiaries and the NA by recognising their project work.
2.4. Use of the monitoring results (NA Reports, Institution Follow-up Reports and NA staff experience):
• To enlarge and improve the pedagogical and technical support given by the NA to beneficiaries and potential project
makers in order to enhance the partnerships’ quality.
• To identify success stories and examples of best practices useful for valorisation purposes (dissemination and exploitation of
results).
• To establish impact assessment indicators.

3 / 35
M-Tool SET

Section II.

M-TOOL STRATEGY FOR MONITORING EDUCATIONAL PARTNERSHIPS

Section II. A

M-TOOL STRATEGY - METHODOLOGY

WORK PHASE PROCEDURE TOOL / OUTCOME


1 Memo with information about the monitoring strategy. NA Memo
Preparation of the
2 Letter to the institution Letter to the
field monitoring
(At least one week before the visit) to inform about the aim and methodology of institution
process
the visit.
3 Monitoring visit in agreement with the principles of qualitative techniques for General Interview
collecting data (interview and observation). Guide
Field work /
Using the General Interview Guide during the visit and carrying out direct
Collecting data
observation of documents and premises.
One day or half-day visit.
4 Guide for analysis/interpretation of monitoring data: Guide for analysis of
ƒ Record the collected data in written and organize them subsequent to the monitoring data
Data analysis and
visit.
interpretation
ƒ Employ a common method to analyse and to interpret the data (Content
Analysis).
Reporting 5 NA Report to the institution with feedback on the monitoring visit. NA Monitoring Report
Follow-up 6 Follow-up Report on the implementation of the NA recommendations. Follow-up Report

4 / 35
M-Tool SET
Section II.B

M-TOOL STRATEGY - TOOLS

1. NA MEMO

TOPICS

• General selection criteria of the institution (regarding the NA Guide orientations).


• Specific selection criteria regarding a concrete project:
o Defining the purpose of the visit (due to difficulties detected, using the criterion ‘visits to coordinators’, to choose a
specific thematic, etc).
o Selecting specific project local activities and mobilities emphasized in the work plan / contract (one or several main
activities; e.g. the more recent, the ones that involved more persons, the one with more visibility, the only ones
implemented…), by using a principle of purposive sampling.
• Basic Data about the project:
o Institution data (identification, type, level, nature, contacts);
o Project basics (code, type, title, countries);
o Project content (concrete aims, target-groups, thematic areas);
o Quality assessment (score and overall comments).
• Definition of NA staff.
• Letter to the institution.

5 / 35
M-Tool SET

2. LETTER TO THE INSTITUTION

TOPICS

• Communicating the objectives, both general (regarding the General Interview Guide) and specific (concerning the specific
selection criteria related to the “case study” i.e. this concrete project).
• Informing about the method of the visit, regarding the use of the group interview.
• Defining the participants in the monitoring visit and their role (institution management, project coordinator, other staff directly
involved, pupils/learners directly involved).
• Request of documentation for observation regarding one or several finalized activities of the work plan (local activities and
agenda of the mobilities).
• Guidance to gather quantifiable data about the level of participation in activities (local, mobilities and
dissemination/valorisation).
• Request for a written confirmation of the availability of the participants on the day of the visit (to ensure the presence of the
relevant participants involved in the project activities and to have an agreement in advance on the agenda and conditions
of the visit).

6 / 35
M-Tool SET

3. QUALITATIVE TECHNIQUES FOR COLLECTING DATA

TOOL FOR COLLECTION DATA INTERVIEW

ƒ Face-to-face verbal interchange by group interviewing.


ƒ Semi-structured (using a general interview guide in a flexible approach):
- Interview questions are planned but not standardized in advance because specific
questions will emerge during the interview;
Type of interview - There are no pre-set response categories.
ƒ Group interview:
- Several people simultaneously (management, project coordinator, project team,
pupils/learners, other participant staff);
- Definition of who will be interviewed, why and how many persons.
ƒ Understanding the project progress.
Purpose ƒ Gathering cumulative data with the group situation.
ƒ Stimulating people to make their views, motives and reasons explicit.
ƒ One-time interview (variable duration).
ƒ Use of the General Interview Guide:
- Use the pre-set questions (Summary of the Interview Guide) to know how to ask and
gather relevant information (related to the objectives);
- Flexible use and order of questions, adapted to the monitoring objectives and project
life cycle;
Strategy
- Make use of the Guiding Topics to manage the answer and avoid deviations from the
objectives.
ƒ The role of NA staff: interviewer as a facilitator, who supplies the questions and topics for
answers and group interaction.
ƒ Managing the interview
- In the NA:

7 / 35
M-Tool SET
TOOL FOR COLLECTION DATA INTERVIEW

ƒ preparing and scheduling the interview;


ƒ reading the application form, the contract, reports (if any) or other documents
related to the project.
- In the institution:
ƒ having the application form and the contract available during the visit;
ƒ beginning the interview with their stated objectives;
ƒ giving oral feed-back during the visit;
ƒ closing the interview with an overall view of the strong and weak points,
including recommendations;
ƒ writing down notes about the participants' speech (during and after the
interview).
Data from the group interviews ƒ Written records of the NA staff

TOOL FOR COLLECTION OF DATA OBSERVATION

ƒ Qualitative approach, not using predetermined categories and classifications.


Approach
ƒ Observation in a natural open-ended way.
ƒ Triangulation between observational data and the interviewees’ discourse to ensure
Purpose different angles.
ƒ Making wider links related to the project implementation.

ƒ Direct observation of:


- Institution premises;
Scope
- Space used for project activities and dissemination;
- Documentary data, like project materials and (final) products.

8 / 35
M-Tool SET
TOOL FOR COLLECTION OF DATA OBSERVATION

ƒ Observing and listening to the institution’s environment.


ƒ Reading different types of documents:
- of different authorship (teachers/trainers, pupils/learners);
- and different availability (closed, restricted or openly published).
ƒ Definition of what will be observed and why, taking into account the objectives of the
Strategy monitoring visit.
ƒ Categories for describing and analysing the observable data (descriptive and focus
categories).
ƒ Key categories to analyse documentary data: credibility, representativeness and meaning
(who writes them, for what purpose, who read them, what is omitted, what is
accomplished).
Data from the observation ƒ Written records of observable data.

9 / 35
M-Tool SET

4. SUMMARY OF THE INTERVIEW GUIDE

CATEGORIES QUESTIONS

How did you know about Comenius/Grundtvig/Leonardo Programmes?


How did you come across the partners for your project?
Conception Was there a project team to begin the work?
How did you collaborate within the partnership to conceive the common project?
When and how did you inform the institution about the project?

How do you communicate among partners?


How do you share the information with all partners?

Communication in What tools do you use to communicate?


the partnership
Which are the communication languages used during the project?
Was there any linguistic preparation needed?
How did you satisfy those needs?

Partnership Local How is the project included in the institution’s work plan?
activities
Which project activities are included in the curriculum of pupils/learners?
Which activities have you already carried out?
Presently, in what activity are you working?
What was the most crowded/popular project activity?
What changed in the partnership planned activities?
Will you be able to complete the planned activities?

10 / 35
M-Tool SET
CATEGORIES QUESTIONS

What kind of common activities have you already carried out with your partners?

Which mobilities have you already carried out?


What changed in your mobility plan?
Can you complete the planned mobilities?
What kind of preparatory activities were developed before the mobility?
Could you quantify the participating staff?
How did you choose staff for mobility?
Could you quantify the participant pupils/learners?

Mobilities How did you choose pupils/learners for mobility?


How did you twin pupils during their foreign stay/ when hosting partners?
How did you involve your pupils/learners in the implementation of the activities during mobility and /or when
hosting partners?
Which was the contribution given by the parents/families to the mobility activities?
Which was the contribution given by the Associated Partners to the mobility activities?
Which activities were developed during mobility and/or partner’s hosting?
How much time was spent in each kind of activity during mobility and/or partner’s hosting?
What kind of follow-up activities were developed after mobility?

Institution’s How many persons are presently involved in the project, including the project team?
involvement
How is the project team formed at the moment?
How often does the project team meet?

11 / 35
M-Tool SET
CATEGORIES QUESTIONS

Which channels does the project team use to inform / guide the other institution’s participants?
How is the institution’s management involved in the project?
Which responsibilities were taken by the relevant staff regarding the project?
In which tasks were pupils/learners involved?

In which tasks were parents / families involved in the project?


Have you involved other institution(s)/organization(s) from the local community?
Community
involvement In which tasks were the institution(s)/organization(s) of the local community involved?
Did they assume the role of associated partners?
Are you planning to involve other institution(s)/organization(s) from the local community?

Which final products have you already finalized?


Outcomes In what stage are you, concerning the common final product(s)?
What are the tasks and roles for each partner institution concerning the final common product(s)?

Project results What knowledge did you gain through the project regarding its thematic area?
What knowledge did you achieve through the project regarding European issues?
What changed in your language skills due to the project?
What changed in your ICT skills due to the project?
What changed concerning the implementation of pedagogical methods due to the project?
What changed in the organization of the institution due to the project?
What changed concerning motivation in learning / teaching?
What changed in the relationships within the institution’s participants due to the project?

12 / 35
M-Tool SET
CATEGORIES QUESTIONS

What changed in the establishment of links with other institutions of the local community due to the project?
What changed concerning pupils/learners school/training success motivated by the project?

How do you evaluate the progress of the project?


Who participates in the evaluation process?
Evaluation What kind of feedback reached the participants?
Which are the results from your evaluation?
What have you changed as a consequence of your evaluation results?

In which events / initiatives did/will you participate to present /make known your project activities/ experiences
and or results?
What kind of information was produced to use in the dissemination activities?

Dissemination What activities did/will you start and implement, on your own initiative, to present / make public the project?
Have you developed any activity in common with other institutions to present/publicize the project?
How many participants were present in each dissemination activity?
How do you intend to continue to spread/share information about your project?

Which activities have you developed to promote the application of your project results by other institutions /
individuals?
Which project results were applied by other institutions and /or individuals?
Valorisation What kind of institutions / group of individuals have directly benefited from your project results?
What project result(s) turned to be the most significant one(s) in terms of being applicable in similar educational
contexts?
Do you consider that your project results are in harmony with the LLP Programme?

13 / 35
M-Tool SET
CATEGORIES QUESTIONS

Which problems/obstacles did you encounter during the project?


Did you share the problems with all the people involved in the project?
Project influences How did you solve the problems detected during the project’s implementation?
How did the mentioned problems change, in some way, the original project?
Which were the most favourable conditions to the project implementation?

Contacts with the Are you satisfied with the contacts you had with your NA?
NA
How do you think they could be improved?

14 / 35
M-Tool SET

5. GENERAL INTERVIEW GUIDE

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To expose the aims of the


visit.
To describe the
methodology used in the
monitoring visit.
To reinforce the
importance of the
Legitimating the To explain the process of a
institution’s collaboration.
interview monitoring visit.
To motivate the
participants in the project
on the importance of the
monitoring process.
To inform about the feed-
back process regarding the
monitoring results.
Internet / NA website
Personnal /
Professional contacts
To recognise how the To identify the channel(s) How did you know about
Conception public has access to the used to obtain information Comenius/Grundtvig/Leonardo Previous participation
Sectorial Programme. about the Programme Programmes? in other European
Programmes
Participation in NA
events

15 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

Participation in a VP
To understand how the or Contact Seminar
contacts are established to Use of an website for
To be familiar with the begin a partnership. How did you come across the partner finding
partner finding procedure. partners for your project?
To comprehend the criteria Networks
to select the partners. Previous partners in a
different project

To understand the initial


To recognize how the motivation for the project.
institution organized Was there a project team to begin
To know who is the staff How, whom
internally the process of the work?
project conception. involved in the project
team.

To identify the tasks of each


partner institution during
the conception process.
To know the measures
To be aware of the How did you collaborate within Common fill-in of the
taken by the partnership
common conception of the partnership to conceive the joint sections of the
members to elaborate the
the partnership’s project. common project? application form
project.
To identify the articulation
between partner
institutions.

16 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To identify the opportunities


given to the institution
community to be involved In teacher/trainer
To know the awareness of in the project conception. meetings, through
When and how did you inform the
the institution community institution’s posters or
To recognise the institution about the project?
about the project. newspaper articles,
involvement of other on a website, etc
persons besides the project
team.
When (every day,
To identify a flowing every week, every
To be acquainted with the communication throughout month)
Communication in communication strategy the project. How do you communicate among Whom (project team,
the partnership implemented among To identify a regular partners? teachers/trainers,
partners. communication throughout management,
the project. pupils/learners, other
staff)
To identify the measures
Written confirmation
undertaken to share How do you share the information
of activities and
information among the with all partners?
decisions
partnership.
To know the tools used to
What tools do you use to Fax, phone, e-mail,
communicate with the
communicate? regular mail, etc
partners.
To be acquainted with the
Which are the communication
languages used to Curricular, non-
languages used during the
communicate with the curricular
project?
partners.

17 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

Why
Who
To recognize the need for Was there any linguistic (teachers/trainers,
linguistic preparation. preparation needed? management,
pupils/learners, other
staff, parents/families)
Using audiovisual
materials, making a
To identify resources used
How did you satisfy those needs? language course,
for linguistic preparation.
hosting a Comenius
Assistant, etc

Partnership Local To be aware of the ways to


How is the project included in the
activities integrate the project in the
institution’s work plan?
institution’s work plan.
To recognize the
integration of the project in To identify the measures
the institution. taken to integrate the Which project activities are
project activities in the included in the curriculum of Which subjects
curriculum of pupils/learners?
pupils/learners.
To verify the Which activities have you already Where, when and
implementation of the carried out? with whom
local activities at the (teachers/trainers,
To identify the degree of
institution and/or in the management,
accomplishment of the
local community. Presently, in what activity are you pupils/learners,
planned activities.
working? parents/families,
associated partners,
other participants)

18 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

How many
participants of each
kind
(teachers/trainers,
To be aware of the most What was the most
management,
visible project activity. crowded/popular project activity?
pupils/learners,
parents/families,
associated partners,
other participants)
To identify changes in the What changed in the partnership
Why
partnership’s work plan. planned activities?
To be aware of the
capacity to accomplish the
partnership’s work plan. Will you be able to complete the Why (enough
To identify the conditions planned activities? resources, time, etc)
that influence the project’s
implementation.
To understand the
What kind of common activities
consistency of the To identify the common
have you already carried out with When and how
cooperative work done in activities in the partnership.
your partners?
the partnership.
When, where and
who
(teachers/trainers,
To verify the To identify the degree of management,
implementation of the Which mobilities have you already
Mobilities accomplishment of the pupils/learners,
mobility activities. carried out?
mobility plan. parents/families,
associated partners,
other participants),
how many
To identify changes in the What changed in your mobility
Why
partnership’s mobility plan. plan?

19 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To be aware of the
capacity to accomplish the
mobility plan.
Can you complete the planned Why (enough
To be aware of the mobilities? resources, time, etc)
conditions that influence
the implementation of
mobilities.
To understand the strategy To be aware of the What kind of preparatory activities
to implement mobility planning process regarding were developed before the Who
activities. mobility activities. mobility?
Number of
To know about the
teachers/trainers,
composition of the Could you quantify the
management,
participant staff in participating staff?
associated partners,
mobilities.
other participants
To identify the criteria for Staff from different
How did you choose staff for
choosing the staff for subjects, staff with
mobility?
mobility. disabilities, etc
To know about the Number of
composition of the Could you quantify the participant pupils/learners,
participant pupils/learners pupils/learners? school level/type of
in mobilities. training, age
Pupils/learners age,
To identify the criteria for school level/type of
How did you choose
choosing the training,
pupils/learners for mobility?
pupils/learners for mobility. pupils/learners with
disability, etc
Applicable to
How did you twin pupils during Comenius Bilateral
To be familiar with the Partnerships
their foreign stay/ when hosting
criteria to twin pupils.
partners? Host families and
reciprocal twining

20 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To know the direct How did you involve your


contribution of pupils/learners in the
pupils/learners to the implementation of the activities
implementation of mobility during mobility and /or when
activities. hosting partners?
To know the direct
contribution of Which was the contribution given
parents/families to the by the parents/families to the
implementation of mobility mobility activities?
activities.
To know the direct
contribution of the Which was the contribution given
Associated Partners to the by the Associated Partners to the
implementation of mobility mobility activities?
activities.
Who
Visiting other
To know the kind of Which activities were developed institutions, time spent
activities developed during during mobility and/or partner’s as trainer or trainee,
mobility. hosting? attending lessons,
working together on
To recognize the the project, time
consistency of the mobility spent for leisure
planned activities. To be aware of the
How much time was spent in each
coherence of the work
kind of activity during mobility
agenda of the mobility
and/or partner’s hosting?
activities.
To identify the follow-up
What kind of follow-up activities
activities consequently to
were developed after mobility?
the mobility.

21 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

Teachers/trainers,
To verify the degree of How many persons are presently
To spot the volume of management,
involvement in project involved in the project, including
institution’s participants. pupils/learners, other
activities. the project team?
staff participants.
To know the composition of Who
How is the project team formed at
the institution’s project How many
the moment?
team. What subjects
To recognize the dynamics To know the work regularity How often does the project team
of the institution’s project of the project team. meet?
team.
To identify the channels “Comenius Corner”
Which channels does the project
used to inform / guide the
team use to inform / guide the E-mail
institution’s participants
Institution’s other institution’s participants? Meetings
about the project.
involvement
Visit other partner
institutions
To be aware of the To identify measures How is the institution’s Links to other
management’s support to regarding management management involved in the institutions (local
the project. support. project? community,
educational
authorities, etc)
To be acquainted with the To categorize the main Which responsibilities were taken
Teachers/trainers and
staff’s involvement in the responsibilities of the staff by the relevant staff regarding the
other institution staff
project. involved in the project. project?
To be aware of the To classify the main tasks of
In which tasks were pupils/learners
pupils’/learners’ the pupils/learners involved Pupils association
involved?
involvement in the project. in the project.
To identify synergies
Community To recognize the
between the institution and In which tasks were parents /
involvement parents’/families’ Parents Association
families, accomplished families involved in the project?
involvement in the project.
through the project.

22 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To identify synergies
between the institution and Have you involved other
Type of institutions/
the local community, institution(s)/organization(s) of the
organizations
accomplished through the local community?
project.
To recognize the official To categorize the main In which tasks were the
relationships between the tasks of the local institution(s)/organization(s) of the
institution and the local community’s involvement. local community involved?
community. To identify the existence of
Did they assume the role of Named in the
an associated partners’
associated partners? application form
network.
To know about the possible Are you planning to involve other
enlargement of community institution(s)/organization(s) from
involvement. the local community?
To identify the degree of
Which final products have you Final products with
accomplishment of the
already finalized? European Logo
planned outcomes.

To verify the To collect evidence of the What


In what stage are you, concerning
implementation of the existence of the common
Outcomes the common final product(s)? Whom
project outcomes. final product(s).
What are the tasks and roles for
To recognize the
each partner institution
complementary of tasks
concerning the final common
within the partnership.
product(s)?
Teachers/trainers,
To understand the results of To know the improvements
What knowledge did you gain pupils/learners,
Project results the project’s in knowledge in the
through the project regarding its management, other
implementation. thematic area of the
thematic area? institution staff,
project.
parents/families, etc

23 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

Teachers/trainers,
To recognize the
What knowledge did you achieve pupils/learners,
improvements made in the
through the project regarding management, other
European Dimension of
European issues? institution staff,
Education/Training.
parents/families, etc
Teachers/trainers,
To know the improvements
pupils/learners,
in language skills achieved What changed in your language
management, other
through the project’s skills due to the project?
institution staff,
implementation.
parents/families, etc
To identify the Teachers/trainers,
improvements pupils/learners,
What changed in your ICT skills
accomplished in ICT skills management, other
due to the project?
through the project’s institution staff,
implementation. parents/families, etc
To know the improvements
in pedagogical methods
with the project’s
implementation. What changed concerning the Teachers/trainers,
implementation of pedagogical management, other
To identify the methods due to the project? institution staff
implementation of new
pedagogical methods due
to the project.
To identify the
improvements in the
management approach. What changed in the organization Teachers/trainers,
To know the new of the institution due to the management, other
organization strategies project? institution staff
implemented due to the
project.

24 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To identify the
improvements on What changed concerning Pupils/learners /
motivation of the project motivation in learning / teaching? teachers/trainers
participants.
Between
To know the improvements teachers/trainers and
in interpersonal pupils/learners,
relationships stimulated by among
the project. What changed in the relationships pupils/learners,
within the institution’s participants among
To identify the due to the project? teachers/trainers,
reinforcement of the between staff in
institution’s internal general, between
cohesion. teachers/trainers and
parents/families
To recognize the
What changed in the
improvements on linking
establishment of links with other
the institution to the local
institutions of the local community
community due to the
due to the project?
project.
To know the improvements
in pupils/learners What changed concerning
school/training pupils/learners school/training
performance due to the success motivated by the project?
project.

25 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To distinguish the
implementation of a two-
level evaluation process At institution level and
(combining the institution at the partnership
level and the partnership level
level). Phases
To identify the evaluation How do you evaluate the progress Méthodes
Evaluation phases. of the project? Techniques
To recognize the existence (interviews,
of an ongoing evaluation questionnaires,
To be aware of the process. observation, etc)
evaluation strategy To know the methods used What
adopted in the project. to collect, analyse and
interpret evaluation data.
Teachers/trainers,
management,
To identify the level of pupils/learners, other
Who participates in the evaluation
participation in the staff, parents/families,
process?
evaluation process. etc
How many
At institution level and
To identify the procedure to What kind of feedback reached
at the partnership
report on evaluation results. the participants?
level

26 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To know the institution’s


evaluation results (self-
evaluation).
To be familiar with the
strong/weak points of the At institution level and
Which are the results from your
project according to the at the partnership
evaluation?
To be aware of the self- institution’s view. level
evaluation results. To know the
recommendations resulting
from the self-evaluation
process.
To identify the adoption of What have you changed as a At institution level and
recommendations made at consequence of your evaluation at the partnership
evaluation reporting stage. results? level
Conferences,
To identify the degree of In which events / initiatives did/will seminars, meetings,
To recognize the exhibitions
development of the accomplishment of the you participate to present /make
Dissemination
dissemination strategy. planned dissemination known your project activities/ Where, when, how, to
activities. experiences and or results? whom
Who presented

To know the tools used to Content of: a


disseminate the project. webpage, a
What kind of information was newspaper, a
To be familiar with the produced to use in the PowerPoint
criteria used to select the dissemination activities? presentation, an
information for exhibition, a
dissemination purposes. performance, etc

27 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To distinguish the nature of


the initiative regarding the What activities did/will you start Auto-initiative
dissemination activities. and implement, on your own Co-authorship with
To identify the measures initiative, to present / make public other institutions
taken to disseminate the the project? (who)
project.
Common
To identify the existence of organization at
cooperative work Have you developed any activity partnership level
regarding the in common with other institutions Shared organization
implementation of to present/publicize the project? with the associated
dissemination activities. partners or other
organizations
Teachers/trainers,
management,
pupils/learners, other
To be aware of the level of How many participants were
staff, representatives
public knowledge about present in each dissemination
from the educational
the project. activity?
authorities, from the
associated partners,
etc
To identify the channels
used to disseminate Spread experiences,
information about the activities and project
project. How do you intend to continue to results
spread/share information about
To know the kind of your project? What, when, where,
institution’s participants how, by who and to
taking part in the whom
dissemination activities.

28 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

Training courses,
To identify the degree of Which activities have you
seminars,
accomplishment of the developed to promote the
conferences,
planned valorisation application of your project results
pedagogical
activities. by other institutions / individuals?
materials’ offers
To identify the kind of
project results more
When, where, how
valuable to be transferred
To understand the transfer Which project results were applied and by whom
to other educational
possibilities of the project contexts. by other institutions and /or At individual, local,
results (apprenticeships individuals? regional or national
and/or final products). To know the extent of the
level.
project results at national /
international scale.
Distinction between
Valorisation
the participants
To identify the target What kind of institutions / group of (internal target
group(s) that could benefit individuals have directly benefited group) and the
from the project results. from your project results? external individuals
How many
What project result(s) turned to be
To recognize how the
To be acquainted with the the most significant one(s) in terms Target group(s) of the
project results are linked to
contribution of the project of being applicable in similar project
the expected impact.
results to achieve Lisbon educational contexts?
objectives and/or the To identify the
operational objectives and achievement of the project Do you consider that your project How
priorities of the LLP results in face of the results are in harmony with the LLP
Programme. objectives and priorities of Programme? Why
the LLP Programme

To be aware of the To identify specific factors


At the institution and
difficulties in project that negatively influenced Which problems/obstacles did you
Project influences at the partnership
development. the project’s encounter during the project?
level
implementation.

29 / 35
M-Tool SET
CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

To know the solving


problem procedures
implemented in the Who (project team,
partnership. management,
Did you share the problems with all
To spot the effectiveness of partnership
the people involved in the
the communication coordinator, all
project?
strategy. partners, NA,
To identify the level of authorities, etc)
cooperation between the
project participants.
To identify concrete How did you solve the problems
solutions in face of the detected during the project’s Who, when
project’s difficulties. implementation?
To be aware of the level of How did the mentioned problems
deviation from the initial change, in some way, the original Application form
project plan. project?
To know specific factors
To be familiar with the Which were the most favourable At the institution and
that positively contribute to
factors in favour of the conditions to the project at the partnership
the project
project development. implementation? level
implementation.
To spot the strong /weak
points in the support given Are you satisfied with the contacts
To be acquainted with the
by the NA to the you had with your NA?
Contacts with the degree of satisfaction of
beneficiary institution.
NA the institution concerning
the NA. To identify ways to improve
How do you think they could be
NA best support and
improved?
guidance.

30 / 35
M-Tool SET

6. GUIDE FOR ANALYSYS OF MONITORING DATA

METHODOLOGICAL
FRAMEWORK TO CONTENT ANALYSIS
QUALITATIVE ANALYSIS
ƒ Establishing how the NA reaches the findings about the monitoring visit.
Purpose ƒ Organization, categorization and interpretation of the information collected (through interview and
observation).
ƒ Coding the data collected in the interview to do their labelling:
- The pre-coded categories will be used to validate the interview responses, because these
categories are based on a clear definition of aims (included in the General Interview Guide).
- For complementing the use of the categories pre-specified (on the General Interview Guide),
other data categories (either descriptive or higher–inference) will be developed if necessary.
- The aim of organizing the data by categories is to reduce data without significant loss of
information, summarizing the data by pulling together themes and identifying patterns and
regularities.
- The data collected is organized by small or large sentences labelled by the categories and
objectives (units of sense), filled in the column “Monitoring Record” of the Analysis Grid.

Strategy ƒ Developing propositions based on the patterns and connections evidenced with the coding, requiring
some degree of inference beyond the data.
- Drawing the findings to establish a meaningful and coherent picture of the project
implementation, putting in evidence the strong and weak points and defining
recommendations (filled in the column “Comments for the Report” of the Analysis Grid
according to the items of the NA Monitoring Report).
- Crossing the data collected in the interview with the observation data to create a
correspondence between verbal responses (what people say) and what they do and what
they say they do.
- The findings will be supported by crossing the observation evidence (from direct observation
and from the interview) with the specific objectives established for each category in the
General Interview Guide.

31 / 35
M-Tool SET
EXAMPLE OF THE ANALYSIS GRID FOR THE INTERVIEW

Categories Objectives Monitoring Record Comments for the Report

Filled in with inferences (findings)


Filled in with topics constituted by
To recognise how the public has obtained by crossing the
“units of sense” from the notes
access to the Sectorial observation evidence with the
collected by the NA staff, during and
Conception Programme. specific objectives defined in the
after the monitoring visit.
General Interview Guide.
To be familiar with the partner
(…) (…)
finding procedure.
Communication in (…) (…) (…)
the Partnership

EXAMPLE OF THE ANALYSIS GRID FOR OBSERVATION

Observed Dimensions Objectives Monitoring Record Comments for the Report

32 / 35
M-Tool SET

7. NA MONITORING REPORT

• Combination of results from data collected directly (observation) and indirectly (interview).
• Possibility given to the institution to refute matters of fact or interpretations stated in the conclusions of the NA Report (existence of
a 1st draft).

Report structure (final version)

Identification of the institution and the project


Objectives of the report phase:
o Good practices emphasis;
o Counselling and advice.
Monitoring visit:
ƒ Introduction (about the visit):
o Date of visit, identification of the persons interviewed (management, coordination,
other staff and pupils/learners), place;
o Context and other relevant circumstantial information;
o Identification of the NA monitoring team.
ƒ Observed data:
o Documents, materials and other aspects observed in situ;
o Evidence of project activities (artefacts, reports, booklets, log books, multimedia
products, etc).
ƒ Findings:
o Results of the content analysis of the interview(s);
o Reference to important observation evidence, stressing the strengths and the

33 / 35
M-Tool SET
Report structure (final version)

weaknesses identified with reference to the pre-defined parameters/categories;


o Identifying problems and their causes;
o Identifying “seeds” of innovation or excellence;
o Conclusion (general appreciation of the state of project implementation; lessons
learned).
ƒ Recommendations
o What shall be done to solve the identified problems? To adjust activities and/or
resources? To improve action? To make the project better?
ƒ Contribution of the project to the LLP Programme (objectives and priorities).
Institution comments about NA conclusions (in case there are any regarding the 1st draft).
Request for a Follow-up Report:
o Information regarding its minimum requirements;
o Establishing a deadline to its submission to the NA.
Date of next visit (optional).

34 / 35
M-Tool SET

8. FOLLOW-UP REPORT

• Aims:
o To ensure that the recommendations have practical consequences in project implementation.
o To make certain that recommendations are implement during the project lifecycle.
o To guarantee improvements in the quality of the project.
• Content:
o Different from institution for institution;
o Open format, chosen by each institution.
o Minimum requirements:
ƒ The institution has to propose concrete follow-up measures focused on the NA recommendations;
ƒ These concrete measures have to be established within a concrete timeframe during project implementation.

35 / 35