Вы находитесь на странице: 1из 31

Understanding and enhancing evidence in higher education careers services

Tristram Hooley (Reader in Career Development)

www.derby.ac.uk/icegs

Welcome and introduction

Who are you? What are you looking for from today? One thing in your practice that you believe is evidence based?

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

What is evidence?

1. Information that informs practice. This might include LMI, research about graduate and employer attitudes etc. 2. The available and publicly accessible body of facts that can support propositions about careers work.
Can you give examples of each that support your practice?

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

Does it work?

Research is specific not general. However there is clear evidence that careers work has a range of impacts.

www.derby.ac.uk/icegs

Key papers

Whiston et al.s (1998) meta-analysis of 47 studies identified impacts across all types of career interventions. Bimrose et als (2008) five-year longitudinal tracking study of 50 career guidance clients found that one-to-one guidance interventions were regarded as useful by clients, and that guidance services can support adults to make successful transitions in a turbulent labour market. Vuori et al.s (2012) paper used a randomised control trial to demonstrate the impact of a group intervention on career management skills. Carey & Dimmitt (2012) found that there was consistent evidence of a positive relationship between well-organised school counselling programmes and the educational outcomes of students. And lots more see the reference list at the end.

www.derby.ac.uk/icegs

Careers work is most effective(1-3)

1. where it is genuinely lifelong and progressive. 2. where it connects meaningfully to the wider experience and lives of the individuals who participate in it. 3. where it is able to recognise the diversity of individuals and to provide services relevant to individual needs.

www.derby.ac.uk/icegs

Careers work is most effective(4-6)

4. where a range of interventions are combined. 5. where it develops of career management skills. 6. where it is holistic and well-integrated into other support services.

www.derby.ac.uk/icegs

Careers work is most effective(7-9)

7. where it is delivered by skilled professionals. 8. where it is based on access to good-quality career information. 9. where it is quality-assured and evaluated to ensure its effectiveness and to support continuous improvement.

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

The evidence base in HE

There have been very limited attempts to measure the impacts of HE careers work. In a new paper (Taylor & Hooley, forthcoming) we find significant impacts on student destinations from both a career management skills programme and a work placement. Other quantitative work has also highlighted the impact of work placements. Other research in the area tends to be descriptive (what is done), conceptual (how can activity be understood) or developmental (what can be improved).

www.derby.ac.uk/icegs

Features of effective practice in HE


Pre-entry provision. Provision of information and resources. Career assessments and tests. One-to-one advice, coaching and counselling services. One-to-many/group interventions. Curriculum-based interventions. Provision of employer engagement opportunities. Provision of work-related and work-based learning. Provision of a framework for reflection. Awards and other mechanisms to recognise and accredit experience. Graduate and alumni services.

www.derby.ac.uk/icegs

Reflection

What in your practice is evidence based? What in your practice is not evidence based? What new evidence would you like? What do you think works and why?

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

What is evaluation?
When people seek to evaluate what they are doing as part of an attempt to learn and improve, they are usually undertaking a formative evaluation, so called because it is undertaken to inform what is done while the activity is still in progress. We would like to find out how to do these things better
When people evaluate to make a judgement on the value of a particular activity and to draw out what has been learnt, it is usually a summative evaluation; so called because it attempts to create a summary of what has been achieved and what the impacts have been. We would like to find out how well these things work

www.derby.ac.uk/icegs

Why evaluate?
Evaluation enables us to:
examine what we do think about how we can improve it

decide on whether it was worth doing


provide others with a summary to help them to understand what was done.

www.derby.ac.uk/icegs

Evaluation as part of reflective practice

www.derby.ac.uk/icegs

Levels of impact

Results Behaviour Learning Reaction

www.derby.ac.uk/icegs

Discussion: Evaluation data

What data do you have which could allow you to make a judgement about the impact of your practice? What level of impact would it describe?
What data could you collect which could allow you to make a judgement about the impact of your practice? What level of impact would it describe?

www.derby.ac.uk/icegs

Common data sources

Surveys Interviews Focus groups Observations Reflective journals Online data

Be clear what you are collecting and for what purpose. Proliferating data types can make analysis more difficult.

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

Research questions
What do you want to find out? Is it possible to answer this question? (At all? Within the resources you have?) What do we already know about this question? (The literature) What data can you use to find out about it? What will this tell you? What wont it tell you? How will you make sense of what you find (analysis)? Who will you tell? (write up) Why?

www.derby.ac.uk/icegs

Write a research question

www.derby.ac.uk/icegs

Overview
What is evidence? What do we know about what works in careers? What do we know about what works in HE careers? Evaluating your work Designing studies and asking good research questions Publish or perish

www.derby.ac.uk/icegs

Publish or perish

Too little evaluation and impact work on careers work is published.


Writing up your evaluation for broader circulation is an important way to support the development of the sector. Self publication Journal publication Partnership with academics Using external consultants

www.derby.ac.uk/icegs

What next?

www.derby.ac.uk/icegs

Useful links

BJGC http://www.ccdf.ca/ccdf/
CCDF http://www.ccdf.ca/ccdf/ CDI http://www.thecdi.net/ ICCDPP http://www.iccdpp.org/ iCeGS http://www.derby.ac.uk/icegs/ NGRF http://www2.warwick.ac.uk/fac/soc/ier/ngrf/ UKCES http://www.ukces.org.uk/

www.derby.ac.uk/icegs

References
Bimrose, J., Barnes, S-A. & Hughes, D. (2008). Adult Career Progression and Advancement: a FiveYear Study of the Effectiveness of Guidance, Coventry/London: Warwick Institute for Employment Research/Department for Innovation, Universities and Skills. Carey, J. & Dimmitt, C. (2012). School counseling and student outcomes: summary of six statewide studies. Professional School Counseling, 16(2): 146-153. Dent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013). Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition. Bristol: HEFCE. Hooley, T., Marriott, J. and Wellens, J. (2012). What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury Academic. Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU. Taylor, A.R. & Hooley, T. (Forthcoming). Evaluating the impact of career management skills module and internship programme within a university business school. British Journal of Guidance and Counselling. Under review. Vuori, J., Toppinen-Tanner, S. & Mutanen, P. (2012). Effects of resource-building group intervention on career management and mental health in work organizations: randomized controlled field trial. Journal of Applied Psychology. 97(2): 273-286. Whiston, S.C., Sexton, T.L. & Lasoff, D.L. (1998). Career intervention outcome: a replication and extension of Oliver and Spokane (1988). Journal of Counseling Psychology, 45(2): 150-165.

www.derby.ac.uk/icegs

Tristram Hooley
Reader in Career Development International Centre for Guidance Studies University of Derby http://www.derby.ac.uk/icegs t.hooley@derby.ac.uk @pigironjoe
Blog at http://adventuresincareerdevelopment.wordpress.com

www.derby.ac.uk/icegs

Оценить